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ABSTRACT 

The extent to which competency-based instruction 
(CBI) has been implemented in postsecondary-1 evel 

vocational-technical education programs and courses throughout Texas 
was assessed. Questionnaires were mailed to the occupational 
education deans/directors of 69 public community and technical 
college campuses. Each dean/director distributed the questionnaires 
to each occupational program chair/leader and one instructor of each 
occupational preparation program. Forty institutions returned 
completed testionnaires . Although more than 90% of the responding 
insti tutioi-s were using competencies from the workplace when 
developing their curricula, less than 407. were using occupational 
competency examinations to determine students* achievement of the 
competencies. The biggest barriers to implementation of CBI were as 
follows: faculty members 1 fears that CBI will make new demands on 
their time, no provision of release time for faculty to develop a CBI 
system format, and few state-provided incentives for development and 
maintenance of CBI. It was recotr ,-nded that the Texas Higher 
Education Coordinating Board esl ihlish and use program standards for 
CBI and conduct or contract for professional development activities 
and/or programs to develop the knowledge and skills needed to 
implement and maintain CBI. (Appended are the first-round Delphi 
scores, sample assessment questionnaires, list of participating 
community and technical colleges, and model for evaluating CBI.) 
(MN) 
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I. INTRODUCTION 



Background 

The use of a competency-based curricula and the delivery of the curricula is 
supported by both mandate and implied mandate. The Technical and Vocationa l 
Program Guidelines of the Texas Higher Education Coordinating Board advocates the 
use of competency-based curricula and instruction. The Guidelines provide the following 
definition: 

Competencv-based education is designed to teach job related clusters of 
skills and knowledge, the mastery of which forms the basis upon which the 
student is evaluated; competency-based programs must possess the 
following characteristics: 

1. Involvement of business and industry in the determination of the 
job competencies and the expected performance level required 
for successful employment within a defined job or cluster of jobs. 

2. Course sequence which allows the mastery of competencies 
leading to the satisfactory performance of all identified 
competencies. 

The above definition clearly directs that the curriculum and instructional content 
of postsecondary occupational preparation programs shall be competency-based. 

Competency-based instruction in technical and vocational education has been 
described as an instructional delivery system that provides a process by which students 
develop measurable performance competencies, specified by business and industry, that 
will assist students in obtaining gainful employment based on their ability to perform in a 
productive manner. 
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Student achievement is measured by demonstration of mastery of competencies- 
rather than performance on written tests and subsequent comparison to the performance 
of other students. Students are held accountable for mastery of each competency and 
may progress at their own rate. 

A review of final reports of program improvement projects funded by the 
Coordinating Board reveals progress has been made in the achievement of developing 
competency-based curricula that adhere to the two characteristics of CBI specified in the 
CB definition of competency-based education above. Other characteristics of 
competency-based instruction include: 

• Performance objectives are developed for the program. There is one 
performance objective developed for each task/competency. In addition, 
enabling objectives are sometimes developed for each performance objective. 

• The student is informed of the required competencies prior to instruction. 

The specific requirements and content of the program, course, and 
performance objective are provided to the student prior to instruction. 

• Student achievement is based on demonstration of masteiy of specified 
competencies. 

• Criterion-referenced testing procedures are used to evaluate student progress 
and performance. 

• Student competency profiles are maintained for purposes of program 
articulation, student application for employment, and permanent records. 

• Learning time is flexible* 

• Learning is guided by feedback. 

A competency-based instructional system is designed for the delivery of a 
9 competency or student-outcome driven curriculum that will: 

A 

2 

ERIC 8 



• Clearly state exactly what is to be learned (specifies what competencies are to 
be mastered); 

• Provide high quality instruction; 

• Assist students to learn one competency well before progressing to the next; 
and 

• Require each student to demonstrate mastery of the competency. 

The concept for implementing a competency-based curriculum using competency- 
based instruction was emphasized by researchers and many state education agencies in 
the 1970s. This emphasis began to decline in the late 70s due to the emphasis of other 
things such as the educational reform movement and the movement to return to the 
"basics." 

With the demands of business and industry for entry-level employees with 
competencies they need, national concern for educational accountability, and the need to 
serve special populations without "diluting" the curriculum, the concept of competency- 
based instruction is receiving considerable attention again. As we have seen with the use 
of buzzwords in other innovations and concepts, it has been popular to talk about 
competency-based instruction with enthusiasm even if it has not been well understood. 
Postsecondary technical and vocational education personnel in texas are moving away 
from the conversational use of the concept and are in the process of moving toward a 
phase of design, actualization, and implementation of a competency-based instruction 
system. 

The participant evaluations of the CBI workshops conducted by Northeast Texas 
Community College attest to value and benefits of using a competency-based instruction 

3 
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system for increasing student achievement in both occupational courses and supporting 
academic courses of the occupational program- 
In the report of the STATE ASSESSMENT, for Section 116 of the Perkins Act, 
prepared by the Texas Higher Education Coordinating Board, there were two 
recommendations that support and can use competency-based instruction in the 
achievement of the two recommendations. The two recommendations made for Criteria 
I actor II: Sequential courses of study leading to both academic and occupational 
competencies are: 

3. Public community colleges and technical institutes should develop a 
periodic review process to ensure that applied basic work skills are 
incorporated into the curriculum of each technical program* 

4. Public community colleges and technical institutes should develop a 
periodic assessment to ensure that competency-based instruction is 
incorporated in all programs. 

The achievement of recommendation "4" stated above will require the 
development of a model that can be used by individual institutions for the periodic 
assessment of competency-based instruction. 

The Carl D. Perkins Vocational and Applied Technology Education Act 
Amendments of 1990, P.L. 101-392, supports the use of competency-based instruction 
(CB1) to meet the requirements for using Perkins funds as shown by the following 
citations: 

In reference to accountability by state and local standards and measures. 

Sec 115. (b), (1) measures of learning and competency gains, including 
student progress in the achievement of basic and more advanced 
academic skills; 

* 
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A great number of individuals have interpreted this requirement to mean that 
"measures of learning and competency gains" pertain only to basic and academic skills. A 
review of the working papers and the agreement of the conference committees of the 
House and Senate reveals the following from the Senate and agreed to by the House: 

includes measures of learning and competency gains in both academic achievement 
and vocational skills competencies;..." 

The implied requirement for a competency-based curriculum and a CBI delivery 

system is shown in Sec. 115, (f) Report, (2): 

An assessment of the validity, predictiveness, and reliability of such standards 
and measures, unbiased to special populations, in the areas of academic 
achievement, vocational skill competencies, employment outcomes, and 
postsecondary continuation and attainment;... 

The definition of "vocational education" stated in PL 101-392 (Perkins Act) 
implies a mandate for a CBI system by this statement in the definition: 

Such programs shall include competency-based applied learning.... 

The TEXAS STATE PLAN FOR VOCATIONAL AND APPLIED 
TECHNOLOGY EDUCATION. FISCAL YEARS 1992-1994 (State Plan) outlines and 
describes how Perkins funds will be used by eligible secondary and postsecondary 
institutions in Texas to make Te::as and the United States more competitive in the world 
economy by developing more fully the academic and occupational skills of all segments of 
the population. 1 



1 Sec. 2 Statement of Purpose, PL 101-392 
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Postsecondary objectives in the State Plan (1992-1993 Goals) that will specifically 
require the implementation of a CBI system are: 

"1., b). defining and teaching competencies which reflect the needs of 
employers;" and 

"5. To integrate academic competencies in all technical-vocational 
programs." 

The Master Plan - a MASTRR PLAN £QB VOCATION AT , AND TECHNICAL 
EDUCATION. 1991 Update of PART II, does not specifically address or advocate 
competency-based instruction. However, there are indications that the Master Plan 
encourages the use of CBI by following statements found under Goal IV: Improve the 
quality of technical education instruction, counseling, management and leadership 
resulting in increased successful outcomes of students.: 

1) "Development of institutional policies on assessment and revision of 
technical education instructional process [IHE];" 

2) "Number and percent of technical students able^to pass competency 
tests relative to skills attained in courses [IHE]." 

The assessment of competency-based instruction will assist the CB in complying 

with Sec. 201. State Programs and State Leadership of PL 101-392. Specifically, they 

will be meeting the requirements for 201.(a) and 201.(b),3, which states: 

"(3) assessment of programs conducted with assistance under this Act, 
including the development of — 

(A) and 

(B) program improvement and accountability with respect to such 
programs. " 
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Based on the above background information, it was determined that there was an 
immediate need for a statewide assessment to determine the extent to which a 
competency-based instructional system has been implemented for the delivery of 
occupational preparation programs in Texas public community and technical colleges. 
Significance of the Study 

In order to make decisions at the state level for continued assistance to public 
postsecondary institutions, the assessment of competency-based instruction must collect 
information that will produce; 

• A list of validated characteristics of a competency-based instructional 
system. 

• A proven process or system for the formative evaluation or assessment of 
competency-based instruction. 

• A list of barriers that inhibit the implementation of competency-based 
instruction. 

• The status of competency-based instruction, on a statewide basis, of 
postsecondary technical/vocational programs and courses. 

• An awareness by faculty and administrators of the acceptable niinimum 
characteristics or criterion for evaluating competency-based instruction. 

• A list of recommendations for improving the status of competency-based 
instruction if the findings from the discrepancy analysis show that there are 
needs for improvement. 

The results of the assessment will have the following uses: 

The CCTC Division staff of the CB will have documented information that 
will assist them in identifying and supporting state leadership activities that 
will assist local institutions in the expansion and improvement of 
competency-based instruction. 



• Local institutions will be able to use the findings of the assessment and 
recommended strategies for improving competency-based instruction at the 
local level. 

• Local institutions can use the developed and piloted assessment 
(evaluation) process to periodically identify what should be done to 
improve comp:tency-based instruction locally and to ensure that 
competency-based instruction is incorporated in all programs. 

Project Goal s and Objectives 

The goal of this project was: 

To design and pilot a statewide assessment process or model that can be used to 
determine the extent of the use of competency-based instruction by all 
technical/vocational education programs and courses in all public community and 
technical colleges. 

The objectives of the project were: 

A. Create and use a project advisory committee with membership representing 
a community college or technical college from each of the Texas Higher 
Education Regions. 

B. Establish characteristics of competency-based instruction that are: 1) 
essential to complying with federal and state mandates and implied 
mandates for competency-based instruction and competency attainment; 
and 2) appropriate for adaptation using current resources of local 
institutions and resources available from the CCTC Division of the 
Coordinating Board. 

C. Design, develop, and pilot a statewide assessment model that will 
determine the status of the use of competency-based instruction in the 
delivery of postsecondary technical/vocational education. 

D. Develop and implement a dissemination plan for publicizing the project 
and distributing the project products. 

E. Evaluate the project in terms of project installation, process, and products 
using internal and external input. 
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IL PROCEDURES 



The primary focus of any educational evaluation (assessment) is to determine the 
status of an educational system or a specific component or process within an educational 
system. Unless specifically defined as to purpose, assessment in education is most often 
used to refer to a process of gathering information for the purpose of making decisions. 

In the development and implementation of an assessment activity or program, the 
planners must define their terms very carefully and specify the purposes of the 
assessment to be conducted. Before any assessment activities are planned, there are two 
important assumptions that should be considered. 

One assumption that can be made is that the planned assessment can obtain 
information that is needed and has not been compiled and is not readily available 
elsewhere. Therefore, the question that became important for this specific assessment 
project is: "Has a methodology been developed and proven to produce the specific 
information that the proposed assessment project is expected to obtain?" 

The second assumption which can be made is that the information obtained be 
the assessment can and will be used by the decision makers to improve the educational 
component or process being assessed. The ultimate design of any assessment is 
dependent upon the specific purpose of the assessment. 

This assessment was designed to determine the extent to which competency-based 
instruction in vocational/technical education programs and courses in all public 
community and technical colleges has been implemented. 



Assessment is formative evaluation. Formation evaluation or assessment is 
normally done by collecting data that is quantitative. The assessment is conducted to 
gather specified information about status and comparing the findings or analyzed status 
information with predetermined goals, objectives, or characteristics of program, 
component, or process being assessed. 

An assessment program or project must include a process for "discrepancy 
analysis" which will pinpoint the specific purpose for which the assessment is made. The 
analysis of information collected by the formative evaluation or assessment must be able 
to identify "what is" and compare the "what is" with the "what should be" or desired status 
that describes the characteristics or measures of a prestated goal or objective. 

Determining the status of CBI in Texas postsecondary technical/vocational 
programs, by assessment, will enable the CCTC Division to develop state programs or 
state leadership activities that will assist the postsecondary institutions in planning and 
implementing a mandated and quality competency-based instructional system throughout 
Texas. 

Since the Texas Higher Education Coordinating Board (CB) had not established 
performance measures and standards for the evaluation of competency-based instruction, 
it was necessary to establish a list of characteristics of a competency-based instruction 
system. 

A listing of required competency-based instruction characteristics were established 
for use as questionnaire items for the assessment. The list of characteristics were 
established using a modified Delphi technique. The results of the final round of the 
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Delphi is presented in appendix A. TT>e required characteristics of a competency-based 
instruction system established by the Delphi were then placed on the assessment 
questionnaire. 

An assessment questionnaire was prepared for mailing to: deans/directors of 
postsecondary occupational education programs; program chairs/leaders; and instructors 
of occupational programs. The questionnaires were mailed to the occupational 
education deans/directors at sixty-nine public community and technical college campuses. 
The dean/director at each campus distributed the questionnaires to each occupational 
program chair/leader and to one instructor of each occupational preparation program. 
An example of the assessment questionnaires mailed to the deans/directors are 
presented in appendix B. Forty community and technical colleges participated in the 
assessment. A list of the community and technical colleges participating in the statewide 
assessment of competency-based instruction is presented hi appendix C. 
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ffl. PRESENTATION OF FINDINGS 



This chapter presents the findings of the statewide assessment of competency- 
based instruction in Texas public two-year community and technical colleges. The 
findings are presented in terms of the perceptions of: the deans/directors of all 
programs; the chairs/leaders of specific programs; instructors of specific programs; 
chairs/leaders across all programs; instructors across all programs; and all respondents to 
the questionnaires. 

Deans/Directors of AL Occupational Programs 
The responses of the deans/directors for all occupational program are shown in 
Table 1. Table 1 reveals that more than fifty-percent of the responding institutions had 
implemented forty-five of the fifty-two characteristics of a competency-based instructional 
system. It was found, based on the responses of the deans/directors, that fifteen of the 
fifty-two of the required characteristics had been implemented by more than ninety 
percent of the respondents. 

Table 1 reveals that responses from the deans/directors placed questionnaire item 
39 as one of the least implemented characteristic. Questionnaire item 39, "A copy of the 
student's competency profile is included in the student's permanent record", had been 
implemented only sixteen percent of the respondents. The required characteristic 
"Students exiting a program are provided with a copy of their competency profile" had 
been implemented by fourteen percent of the respondents. 

The respondents were requested to rate the degree of implementation of the fifty- 
two characteristics on a scale of "1" to "5" with "1" being a minimum implementation 
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TABLE 1 



Implementation of Competency-Based Instruction Across AH Program 
Areas as Perceived by Deans/Directors 
of Postsecondary Occupational Education Programs 



Characteristics: 

1 Occupational competencies are obtained from the 
workplace. 

2 Academic, competencies needed to develop and/or 
perform the occupational com H ocencies are obtained 
from the workplace. 

3 The occupational competencies are sequenced for 
instructional purposes. 

4 The occupational competencies are clustered for course 
development. 

5 Occupational program courses are named and described. 

6 The prerequisite courses are identified and listed. 

7 The support courses are identified and listed. 

8 Technical and support (including academic courses) are 
arranged so that sequential mastery of competencies 
leads to the satisfactory performance of all competencies 
of the workplace for the occupation(s) being prepared 
for by the student. 

9 Legitimate or validated career ladder exit points are 
identified in the curriculum. 

10 The curriculum is kept current with the requirements of 
the workplace by the use of a program advisory 
committee composed of representatives of the 
workplace. 

1 1 Occupational competency profiles are developed for the 
occupational curriculum (program). 



Percent of 

Respondents Degree of 
Implementing Implementation 
this 1 = minimum 

Characteristic 5 = maximum 



97.30% 



70.27% 9 

94.59% 

89.19% 
100.00% 
97.30% 
97.30% 



97.30% 



86.49% 



100.00% 



72.97% 



3 
4 
4 
4 
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TABLE 1 



Characteristics: 

12 Objectives are written in performance terms for each 
occupational competency, 

13 Objectives are written in performance terms for each 
occupational task, 

14 Enabling objectives are written for each task 
performance objective. 

15 Instructional materials, equipment, and supplies essential 
for performance of the competency are available for 
instructor and student use. 



Percent of 

Respondents Decree of 

Implementing Implementation 

|hi£ 1 = minimum 

Characteristic 5 = maximum 



97.30% 



81,08% 



56.76% 



97.30% 



16 



17 



18 



19 
20 
21 

v 22 
. 23 



Learning activities are designed to support the students' 
ability to learn and perform each competency 
successfully 

All objectives of the programs (each course, each 
competency, and each lesson) are written in 
performance terms. 

Students entering the occupational program (curriculum) 
are routinely tested prior to entering the program for 
communication, computational, and current occupational 
skills/aptitudes. 

Instructional content of the course(s) is derived from an 
analysis of tasks validated in the workplace. 

The student is informed of the required competencies 
and/or tasks prior to instruction. 

Learning time is flexible 

Learning is guided by feedback 

Students are informed of performance measures in 
advance of instruction for each task 



94,59% 



78,38% 



78.38% 

89,19% 

94.59% 
62.16% 
94,59% 

83.78% 



3 
2 
3 
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TABLE 1 



Characteristics. 



24 



26 



27 
28 

29 

30 
31 

32 
33 
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Students are informed of performance measures in 
advance of instruction for each competency. 

Each student is required to perform each task at a 
specified standard before receiving credit or moving on 
to the next task. 

Each student is required to demonstrate mastery of 
competency at a standard specified by the workplace 
before receiving credit or moving on to the next 
competency. 

Students are required to perform each task in a joblike 
setting. 

A clear specification of student achievement is provided 
to the student in performance terms prior to each 
learning experience. 

The delivery of instruction or learning experiences 
provides for immediate and frequent feedback from the 
student. 

The instructor is a manager and facilitator of learning. 

The delivery of instruction is appropriate for the 
different learning styles of students 

A record system is maintained for 

recording and documenting tasks and competencies 

which have been achieved by each student. 

The record system contains data that includes program 
standards and performance measures that are used in the 
continuous evaluation of the 

effectiveness of instruction in terms of student progress 
and competencies mastered 

Task and competency progress records are maintained 
for each student 
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Percent of 

Respondents Degree of 

Implementing Implementation 

thjs 1 = minimum 

Characteristic 5 = maximum 



89.19% 



72.97% 



56.76% 
64.86% 

81.08% 

91.89% 
97.30% 

91.89% 
70.27% 



56.76% 



56 76% 



2 
3 



TABLE 1 



Characteristics: 

35 Student progress records are current. 

36 Student progress is determined by criterion-referenced 
measurement. 

37 Competency Profiles are kept current for each student. 

38 The minimum acceptable measure occupational 
competency or task achievement is based on the 
performance level established by the workplace. 

39 A copy of the students' competency profile is included 
in the students' permanent records. 

40 Students exiting a program are provided with a copy of 
their competency profile. 

41 Time required for a student to master a competency is 
not consideied in rating or grading a student if flexible 
learning time is available. 

42 Each student is graded on his or her own level of 
achievement based on predetermined standards and not 
in comparison to other students. 

43 Grades for competencies achieved are not lowered by 
competencies not achieved if flexible learning time is 
available. 

44 The grading system provides for achievement 
(proficiency) above the established performance 
(minimum) level. 

45 Performance levels (criterion) for each occupational 
competency of the program (course) are obtained from 
the workplace. 

46 The competency exams are totally performance 
measured 



Percent of 

Respondents Degree of 

Implementing Implementation 

this 1 « minimum 

Characteristic 5 = maximum 

78.38% 3 



70.27% 2 

40.54% 2 

78.38% 2 

16.22% 2 

13.51% 2 

45.95% 2 

64.86% 2 

32.43% 2 

72.97% 3 

70.27% 3 

37.84% : 
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TABLE 1 



Characteristics: 



Percent of 

Respondents Degree of 

Implementing Implementation 

this I = minimum 

Characteristic 5 ~ maximum 



47 Student mastery of occupational competencies are 

determined (assessed) on an individual basis. 75.68% 

48 Competency exams approved by the workplace are used 
in determining a student's mastery of an occupational 

competency. 40.54% 

49 Criteria for passing performance tests are acceptable for 

on-the-job performance. 78.38% 

50 The time taken to complete performance tests is 

acceptable for the occupation (workplace) 75 68% 

51 In an articulated program or a tech-prep curriculum, 
both secondary and postsecondary instructors have 
accepted the criteria or standards established by the 

workplace to determine mastery of each competency. 59.46% 

52 The same criteria stated in the instructional objective are 

used in the performance test (exam). 78.38% 



ERLC 



17 



and "5" being a maximum rating of implementation. The ratings of the individual 
respondents were tabulated to provide for a statewide mean. None of the fifty-two 
required characteristics had been implemented on a statewide basis with a mean rating 
of "5." Four of the required character sties had been implemented on a statewide basis 
at a mean rating of "4." 

Twenty-two of the required characteristics were perceived to have been 
implemented by the deans/directors statewide at a rating of "3." Twenty-six of the 
required characteristics of a competency-based instruction system were perceived to have 
been implemented statewide at a degree of "2." 

A review of Table 1 reveals that all of the responding institutions had 
implemented the required characteristic "The curriculum is kept current with the 
requirements of the workplace by the use of a program advisory committee composed of 
representatives of the workplace." Table 1 further reveals that this characteristic had a 
mean implementation of eighty percent based on the perception of the responding 
deaas/directors of occupational education programs. 
Barriers to Implementation 

Three types of barriers to the implementation of a competency-based instructional 
system were included on the questionnaire. Items on the questionnaire that were related 
to organization were coded with a "0 " Items considered as interpersonal barriers were 
coded with an "I." Material barriers items were coded with an "M." 

Table 2 provides the tabulation of the responding deans/directors to the questions 
on barriers. The greatest barrier to the implementation of a competency-based 

18 
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TABLE 2 



Barriers to the Implementation of Competency-Based Instruction 
Across All Program Areas as Perceived by Deans/Directors 
of Postsecondary Occupational Education Programs. 



Barriers : 

0- 1 Faculty are not provided release time needed to develop 
CBI system format. 

0- 2 These institutions do not have specialists trained in CBI 
systems to assist faculty. 

0- 3 Students are expected to complete occupational 

preparation programs (or courses) in a specified number 
of hours. 

0- 4 Traditional college procedures do not permit students to 
enter and complete programs with the flexibility 
possible with a CBI system. 

0- 5 It is difficult to schedule students and facilities needed 
when using a CBI system. 

0- 6 Students may require greater time to master 
competencies in a CBI system. 

0- 7 CBI programs with open entry/open exit do not match 
institutional methods for determining funding. 

0- 8 Faculty have little access to clerical help needed for CBI 
development, implementation, and maintenance. 

0- 9 The staff of the Community and Technical College 
Division of the Coordinating Board has not reached 
consensus about Competency-Based Instruction. 

0-10 Guidelines and policies of the Texas Higher Education 
Coordinating Board communicate mixed messages to 
local institutions of the Coordinating Boards 
commitment and direction relative to CBI 



Percent Of 
Percent of Respondents 
Respondents Who Did Not 
Agreeing This Know If This 
Is A Barrier Was A Barrier 



72.97% 



64.86% 



54.05% 



70.27% 



43.24% 



48.65% 



62.16% 



62.16% 



51.35% 



2.70% 



8.11% 



8.11% 



8.11% 



8.11% 



24.32% 



21.62% 



8.11% 



35.14% 



64.86% 



16.22% 
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TABLE 2 



Barriers: 



Percent Of 
Percent of Respondents 
Respondents Who Did Not 
Agreeing This Know If This 
Is A Barrier Was A Barrier 



0-11 There is a lack of a comprehensive State plan and 
methodology for the implementation of CBI. 

0-12 Local administrators do not provide sufficient supportive 
resources needed to encourage faculty participation in 
CBI. 



67.57% 



59 46% 



18.92% 



2.70% 



0-13 Faculty are not aware of opportunities to learn the skills 
needed to implement CBI. 



48.65% 



13.51% 



0-14 CBI requires students who are mature and self-directed 
and who can assume responsibility for his/her own 
learning or occupational skills. 



48.65% 



16.22% 



I- 1 Difficulty in getting faculty and institution to accept 
demands of CBI characteristics. 



70.27% 



8.11% 



I- 2 Many inservice, workshops, and conferences are not 
competency-based. 



62.16% 



16.22% 



I- 3 State approved teacher education programs are not 
co m p eten c y -b as td . 



59.46% 



35.14% 



1- 4 CBI system not perceived by faculty as an improvement 
over the existing system for delivery of instruction. 



59.46% 



21.62% 



I- 5 Faculty have insufficient knowledge of CBI. 



67.57% 



8.11% 



I- 6 Faculty believe CBI will .make new demands on their 
time. 



86.49% 



5.41% 



I- 7 Faculty express a negative attitude about CBI and fail to 
conceptually support its development and/or fail to 
demonstrate and/or commit to goals of CBI. 56.76% 

I- 8 Administrators express a negative attitude about CBI 
and fail to conceptually support its development and/or 
fail to demonstrate and/or commit to goals of CBI. 16 22% 



18.92% 



18.92% 
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TABLE 2 



Barriers ' 

I- 9 Faculty have responsibility for CBI program 
deveIopment,but lack skill. 

1-10 Teacher education programs do noi adequately prepare 
faculty to use a CBI system. 

I- II Faculty fears that the CBI system may result in the loss 
of his/her position. 

M- 1 Institutions find it difficult to obtain/maintain 
appropriate equipment required for CBI. 

M- 2 Institutions have difficulty in keeping CBI instructional 
materials current. 



Percent Of 
Percent of Respondents 
Respondents Who Did Not 
Agreeing This Know If This 
Is A Barrier Was A Barrier 



62.16% 



67.57% 



8.11% 



54.05% 



59.46% 



5.41% 



27.03% 



29.73% 



16.22% 



10.81% 



M- 3 There are few, if any, state-provided incentives to 

institutions' administrators and faculty for implementing 

and maintaining a CBI system. 78.38% 

M- 4 Facilities are not appropriate for CBI. 35.14% 

M- 5 Printing, storing, and distributing individualized 

instructional materials require greater resources than do 

conventional programs. 72.97% 

M- 6 CBI programs require more advanced teaching 

technology that is difficult to obtain and maintain. 59.46% 

M- 7 State funding for educational instruction is not sufficient 

for CBI programs. 75.68% 



16.22% 
10.81% 

8.11% 
10.81% 
16.22% 
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instruction system (CBI) as perceived by eighty-six percent of the responding 
deans/directors was the interpersonal barrier "1-6 Faculty believe that CBI will make 
new demands on their time." 
* The greatest organizational barrier to CBI implementation reported by the 

deans/directors (73%) was "0-1 Faculty are not provided release time needed to 
develop a CBI system format." 

The greatest material barrier shown on Table 2 was "M-3 There are few, if any, 
state-provided incentives to institutions' administrators and faculty for implementing and 
maintaining a CBI system." 

All Respondents Across All Program Areas 
Percent of Implementation 

Table 3 shows a comparison of the perception by type of respondents for the 
implementation of the required characteristics of a competency-based instruction system 
across all program areas. A review of table 3 reveals there is close agreement of 
implementation by the three categories of respondents for most of the required 
characteristics of a competency-based instruction system. 

Table 4 presents the number of responding institutions implementing CBI and 
degree of implementation by vocational program area. The four program areas that had 
the greatest degree of implementation (80%) as reported by the responding institutions 
were: Personal and Miscellaneous Services, Construction Trades, Transportation and 
Material Moving Workers, and Health Professions and Related Sciences. 
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Barriers to CBI Implementation 

The barriers to implementing a competency-based instruction system are 
presented in Table 5, Table 5 presents a comparison of the perception to the barriers 
made by the Deans/Directors, Program Chairs, and Instructors, A review of Table 5 
reveals that, overall, a greater number of Deans/Directors agreed that the statements 
were barriers than did the Chairs and Instructors, Fo. example, a greater number of 
Deans/Directors perceived the statement "0-12 Local administrators do not provide 
sufficient resources needed to encourage faculty participation in CBI" than did Chairs or 
Instructors, The barriers to the implementation have been ranked in descending order 
on Table 6 based on the combined perceptions of all of the respondents. Table 6 reveals 
that the greatest barrier perceived by the respondents was the interpersonal barrier 
"Faculty believe CBI will make new demands on their time." The interpersonal barrier 
"1-5 Faculty have insufficient knowledge of CBI" was in the upper quarter of the ranked 
barriers. 
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TABLE 3 



Comparison of Responses Across All Program Areas By Type 
of Respondents For Implementation of CBL Required Characteristics 



Characteristics 

1 Occupational competencies are 
obtained from the workplace, 

2 Academic competencies needed to 
develop and/or perform the occupa- 
tional competencies are obtained 
from the workplace. 

3 The occupational competencies are 
sequenced for instructional purposes. 

4 The occupational competencies are 
clustered for course development. 

5 Occupational program courses are 
named and described* 

6 The prerequisite courses are 
identified and listed. 

7 The support courses are identified 
and listed. 

8 Technical and support (including 
academic courses) are arranged so 
that sequential mastery of 
competencies leads to the satisfactory 
performance of all competencies of 
the workplace for the occupation(s) 
being prepared for by the student. 

9 Legitimate or validated career ladder 
exit points are identified in the 
curriculum. 



Percent of Implementation 
Across All Program Areas 

Deans/ Program 



Directors Chairs Instructors 

97 92 91 

70 80 78 

94 93 89 

89 89 87 

100 98 96 

97 95 92 

97 90 86 

97 94 90 

86 61 56 
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TABLE 3 



Percent of Implementation 
Across All Program Areas 



Deans/ Program 
Characteristics Directors Chairs Instructors 

10 The curriculum is kept current with 100 98 94 

the requirements of the workplace by 
the use of a program advisory 
committee composed of 
representatives of the workplace. 



11 Occupational competency profiles are 73 67 65 
developed for the occupational 

curriculum (program). 

12 Objectives are written in 97 89 86 
performance terms for each 

occupational competency. 

13 Objectives are written in 81 82 81 
performance terms for each 

occupational task. 



14 Enabling objectives are written for 57 68 64 
each task performance objective. 

15 Instructional materials, equipment, 97 95 89 
and supplies essential ftr 

performance of the competency are 
available for instructor and student 
use. 



16 Learning activities are designed to 95 97 94 

support the students' ability to learn 
and perform each competency 
successfully. 



17 All objectives of the programs (each 78 77 76 

course, each competency, and each 
lesson) are written in performance 
terms. 
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TABLE 3 



Characteristics 

18 Students entering the occupational 
program (curriculum) are routinely 
tested prior to entering the program 
for communication, computational, 
and current occupational 
skills/aptitudes, 

19 Instructional content of the course(s) 
is derived from an analysis of tasks 
validated in the workplace, 

20 The student is informed of the 
required competencies and/or tasks 
prior to instruction, 

21 Learning time is flexible, 

22 Learning is guided by feedback, 

23 Students are informed of 
performance measures in advance of 
instruction for each task, 

24 Students are informed of 
performance measures in advance of 
instruction for each competency, 

25 Each student is required to perform 
each task at a specified standard 
before receiving credit or moving on 
to the next task, 

26 Each student is required to 
demonstrate mastery of competency 
at a standard specified by the 
workplace before receiving credit or 
moving on to the next competency. 



Percent of Implementation 
Across All Program Areas 

Deans/ Program 



Directors Chairs Instructors 

78 59 56 

89 80 83 

95 92 91 

62 61 65 

95 96 93 

83 84 85 

89 85 85 

73 64 67 

57 56 58 



ERLC 



26 



TABLE 3 



Characteristics 

27 Students are required to perform 
each task in a joblike setting. 

28 A clear specification of student 
achievement is provided to the 
student in performance terms prior 
to each learning experience. 

29 The delivery of instruction or 
learning experiences provides for 
immediate and frequent feedback 
from the student. 

30 The instructor is a manager and 
facilitator of learning. 

31 The delivery of instruction is 
appropriate for the different learning 
styles of students. 

32 A record system is maintained for 
recording and documenting tasks and 
competencies which have been 
achieved by each student. 

33 The record system contains data that 
includes program standards and 
performance measures that are used 
in the continuous evaluation of the 
effectiveness of instruction in terms 
of student progress and competencies 
mastered. 

34 Task and competency progress 
records are maintained for each 
stude »t. 

35 Student progress records are current. 



Percent of Implementation 
Across All Program Areas 



Deans/ 
Directors 

65 



81 



92 



97 



92 



70 



57 



57 



78 



Program 
Chairs 

68 



73 



Instructors 
68 

74 



91 



93 



96 
91 

72 



98 
93 

73 



62 



61 



68 



81 



65 



78 
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TABLE 3 



Percent of Implementation 
Across All Program Areas 

Deans/ Program 
Characteristics Directors Chairs Instructors 



36 Student progress is determined by 
criterion-referenced measurement. 

37 Competency Profiles are kept current 
for each student. 

38 The minimum acceptable measure 
occupational competency or task 
achievement is based on the 
performance level established by the 
workplace. 

39 A copy of the students' competency 
profile is included in the students' 
permanent records. 

40 Students exiting a program are 
provided with a copy of their 
competency profile. 

41 Time required for a student to 
master a competency is not 
considered in rating or grading a 
student if flexible learning time is 
available. 

42 Each student is graded on his or her 
own level of achievement based on 
predetermined standards and not in 
comparison to other students. 

43 Grades for competencies achieved 
are not lowered by competencies not 
achieved if flexible learning time is 
available. 



70 73 72 

41 50 48 

78 70 70 

16 29 34 

13 16 17 

46 43 44 

65 82 79 

32 42 44 



TABLE 3 



Characteristics 

44 The grading system provides for 
achievement (proficiency) above the 
established performance (minimum) 
level 

45 Performance levels (criterion) for 
each occupational competency of the 
program (course) are obtained from 
the workplace, 

46 The competency exams are totally 
performance measured* 

47 Student mastery of occupational 
competencies are determined 
(assessed) on an individual basis, 

48 Competency exams approved by the 
workplace are used in determining a 
student's mastery of an occupational 
competency* 

49 Criteria for passing performance tests 
are acceptable for on-the-job 
performance, 

50 The time taken to complete 
performance tests is acceptable for 
the occupation (workplace), 

51 In an articulated program or a tech- 
prep curriculum, both secondary and 
postsecondary instructors have 
accepted the criteria or standards 
established by the workplace to 
determine mastery of each 
competency. 



Percent of Implementation 
Across All Program Areas 

Deans/ Program 



Directors Chairs Instructors 

73 78 80 

70 70 70 

38 47 56 

76 87 81 

41 40 39 

78 77 74 

76 74 73 

59 52 47 
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TABLE 3 



Percent of Implementation 
Across All Program Areas 

Deans/ Program 
Characteristics Directors Chairs Instructors 

52 The same criteria stated in the 

instructional objective are used in the 78 87 83 

performance test (exam). 
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TABLE 4 

Implementation Of CBI By Program Area 



Degree of 

N=40 Number of Implementation 

Institutions As Perceived by 

Program Area Title: Implementing CBI Instructors 

Agricultural Business and Production 19 60% 

Agricultural Sciences 1 40% 

Conservation and Renewable Natural 3 60% 
Resources 

Architecture and Reined Programs 1 60% 

Marketing Operations/Marketing and 22 60% 
Distribution 

Communications 8 60% 

Communications Technologies 6 40% 

Computer and Information Sciences 40 60% 

Personal and Miscellaneous Services 17 80% 

Education 3 60% 

Engineering-Related Technologies 40 60% 

Home Economics 1 20% 

Vocational Home Economics 20 60% 

Technology Education/Industrial Arts 1 60% 

Law and Legal Studies 11 20% 

English Language and Literature/Letters 1 20% 

Parks, Recreation, Leisure and Fitness Studies 4 20% 

Science Technologies 1 60% 
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TABLE 4 



Program Area Title: 

Protective Services 

Public Administration and Services 

Construction Trades 

, Mechanics and Repairers 

Precision Production Trades 

Transportation and Materials Moving Workers 

Visual and Performing Arts 

Health Professions and Related Sciences 

Business Management and Administrative 
Services 



N=40 Number of 
Institutions 
Implementing CBI 

34 

3 

15 

40 

40 

5 

18 

40 

40 



Degree of 
Implementation 
as Perceived by 
Instructors 

60% 

60% 

80% 

60% 

60% 

80% 

60% 

80% 

60% 
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TABLE 5 



Comparison of Responses Across All Program Areas 
Of Barriers To The Implementation Of CBI By Type Of Respondent 



Barriers: 
Organizational 

10- 1 Faculty are not provided release 
time needed to develop CBI system 
format 

0- 2 These institutions do not have 
specialists trained in CBI systems 
to assist faculty. 

0- 3 Students are expected to complete 
occupational preparation programs 
(or courses) in a specified number 
of hours. 

0- 4 Traditional college procedures do 
not permit students to enter and 
complete programs with the 
flexibility possible with a CBI 
system. 

0-5 It is difficult to schedule students 
and facilities needed when using a 
CBI system. 

0- 6 Students may require greater time 
to master competencies in a CBI 
system. 

0- 7 CBI programs with open 

entry/open exit do not match 
institutional methods for 
determining funding. 



Percent of Respondents 
Identifying Barriers to CBI 

Deans/ Program 



Directors Chairs Instructors 

73 60 57 

65 49 39 

54 52 47 

70 60 46 

43 47 41 

49 55 51 

62 49 39 
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TABLE 5 



Barriers; 

0- 8 Faculty have little access to clerical 
help needed for CBI 
development,implementation, and 
maintenance. 

0- 9 The staff of the Community and 
Technical College Division of the 
Coordinating Board has not 
reached consensus about 
Competency-Based Instruction. 

0-10 Guidelines and policies of the 
Texas Higher Education 
Coordinating Board communicate 
mixed messages to local institutions 
of the Coordinating Board's 
commitment and direction relative 
to CBI. 

0-11 There is a lack of a comprehensive 
State plan and methodology for the 
implementation of CBI. 

0-12 Local administrators do not 
provide sufficient supportive 
resources needed to encourage 
faculty participation in CBI. 

0-13 Faculty are not aware of 

opportunities to learn the skills 
needed to implement CBI. 

0-14 CBI requires students who are 
mature and self-directed and who 
can assume responsibility for 
his/her own learning or 
occupational skills. 



Percent of Respondents 
Identifying Barriers to CBI 

Deans/ Program 



Directors Chairs Instructors 

62 ' 69 61 

51 35 29 

65 45 35 

68 48 43 

59 47 42 

49 44 43 

49 47 49 
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TABLE 5 



Barriers: 

I- 1 Difficulty in getting faculty and 
institution to accept demands of 
CBI characteristics. 

I- 2 Many inservice, workshops, and 
conferences are not competency- 
based. 

I- 3 State approved teacher education 
programs are not competency- 
based. 

I- 4 CBI system not perceived by 

faculty as an improvement over the 
existing system for delivery of 
instruction. 

I- 5 Faculty have insufficient knowledge 
of CBI. 

I- 6 Faculty believe CBI will make new 
demands on their time. 

I- 7 Faculty express a negative attitude 
about CBI and fail to conceptually 
support its development and/or fail 
to demonstrate and/or commit to 
goals of CBI. 

I- 8 Administrators express a negative 
attitude about CBI and fail to 
conceptually support its 
development and/or fail to 
demonstrate and/or commit to 
goals of CBI. 



Percent of Respondents 
Identifying Barriers to CBI 

Deans/ Program 



Directors Chairs Instructors 

70 53 40 

62 55 49 

59 36 31 

59 55 43 

68 59 54 

86 73 64 

57 49 36 

16 21 18 
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TABLES 



Percent of Respondents 
Identifying Barriers to CBI 

Deans/ Program/ 
Barriers: Directors Chairs Instructors 

I- 9 Faculty have responsibility for CBI 62 54 45 

program development, but lack 
skills. 

MO Teacher education programs do 68 50 48 

not adequately prepare faculty to 
use a CBI system. 

Ml Faculty fears that the CBI system 8 10 10 

may result in the loss of his/her 
position. 



Material Barriers 

M- 1 Institutions find it difficult to 
obtain/maintain appropriate 
equipment required for CBI. 

M- 2 Institutions have difficulty in 

keeping CBI instructional materials 
current. 

M- 3 There are few, if any, state- 
provided incentives to institutions' 
administrators and faculty for 
implementing and maintaining a 
CBI system. 

M- 4 Facilities are not appropriate for 
CBI. 

M- 5 Printing, storing, and distributing 
individualized instructional 
materials require greater resources 
than do conventional programs. 



54 49 46 

59 49 41 

78 53 41 

35 28 29 

73 66 63 
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TABLE 5 



Percent of Respondents 
Identifying Barriers to CBI 

Deans/ Program/ 
garngrs; Directors ChailS Instructors 



M- 6 CBI programs require more 59 41 35 

advanced teaching technology that 
is difficult to obtain and maintain* 

M- 7 State funding for educational 76 56 49 

instruction is not sufficient for CBI 
programs. 
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TABLE 6 

Rank of Barriers To CBI As 
Perceived By All Respondents 

Bailkis; Mean Percent 

I- 6 Faculty believe CBI will make new demands on their 74 
time. 

M- 5 Printing, storing, and distributing individualized 67 
instructional materials require greater resources than 
do conventional programs. 

0- 8 Faculty have little access to clerical help needed for 64 
CBI development, implementation, and maintenance. 

0- 1 Faculty are not provided release time needed to 63 

develop CBI system format. 

1- 5 Faculty have insufficient knowledge of CBI. 60 

M- 7 State funding for educational instruction is not 60 
sufficient for CBI programs. 

0- 4 Traditional college procedures do not permit students 59 

to enter and complete programs with the flexibility 
possible with a CBI system. 

M- 3 There are few, if any, state-provided incentives to 57 
institutions' administrators and faculty for 
implementing and maintaining a CBI system. 

M0 Teacher education programs do not adequately 55 
prepare faculty to use a CBI system. 

1- 2 Many inservice, workshops, and conferences are not 55 

competency-based. 

I- 1 Difficulty in getting faculty and institution to accept 54 
demands of CBI characteristics. 

I- 9 Faculty have responsibility for CBI program 54 
development,but lack skill. 
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TABLE 6 



Barriers: Mean Percent 

0- 11 There is a lack of a comprehensive State plan and 53 

methodology for the implementation of CBL 

1- 4 CBI system not perceived by faculty as an 52 

improvement over the existing system for delivery of 
instruction. 

0- 6 Students may require greater time to master 52 
competencies in a CBI system 

0- 2 These institutions do not have specialists trained in 51 
CBI systems to assist faculty. 

0- 3 Students are expected to complete occupational 51 
preparation programs (or courses) in a specified 
number of hours. 

0- 7 CBI programs with open entry/open exit do not match 50 
institutional methods for determining funding. 

M- 1 Institutions find it difficult to obtain/maintain 50 
appropriate equipment required for CBI. 

M- 2 Institutions have difficulty in keeping CBI instructional 50 
materials current. 

0-12 Local administrators do not provide sufficient 49 
supportive resources needed to encourage faculty 
participation in CBI. 

0-10 Guidelines and policies of the Texas Higher Education 48 
Coordinating Board communicate mixed messages to 
local institutions of the Coordinating Board's 
commitment and direction relative to CBI. 

0- 14 CBI requires students who are mature and self- s 48 

directed and who can assume responsibility for his/her 
own learning or occupational skills. 

1- 7 Faculty express a negative attitude about CBI and fail 47 

to conceptually support its development and/or fail to 
demonstrate and/or commit to goals of CBL 
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TABLE 6 



Barriers: Mean Percent 

0-5 It is difficult to schedule students and facilities needed 44 
when using a CBI system. 

0- 13 Faculty are not aware of opportunities to learn the 45 

skills needed to implement CBI. 

M- 6 CBI programs require more advanced teaching 45 
technology that is difficult to obtain and maintain. 

1- 3 State approved teacher education programs are not 42 

competency-based. 

0- 9 The staff of the Community and Technical College 38 

Division of the Coordinating Board has not reached 
consensus about Competency-Based Instruction. 

M- 4 Facilities are not appropriate for CBI. 31 

1- 8 Administrators express a negative attitude about CBI 18 

and fail to conceptually support its development 
and/or fail to demonstrate and/or commit to goals of 
CBI. 

1-11 Faculty fears that the CBI system may result in the 9 
loss of. his/her position. 
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IV. SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS 

Summary of Findings 

The goal of this project was to design and pilot a statewide assessment process or 
model that could be used to determine the extent of the use of competency-based 
instruction by all technical/vocational programs and courses in a Texas public community 
and technical colleges. A copy of the model is presented in Appendix D. The data 
presented in Chapter III of this report was the result of the pilot of the statewide 
assessment model. 

By a review of the presentation of the data offered by the pilot, it was found that 
there was some degree of implementation of a competency-based instruction system by 
all of the program areas of the responding institutions. Only four of the program areas 
had an implementation rate of eighty-percent. Seventeen of the programs had an 
implementation rate of sixty-percent and two of the program areas had an 
implementation rate of forty percent. 

Seven of the eleven interpersonal barriers were identified by more than fifty- 
percent of the respondents as barriers to the implementation of a competency-based 
instructional system. 

Although more than ninety percent of the responding institutions were developing 
the curriculum using occupational competencies from the workplace, less than forty 
percent were using occupational competency exams to determine a student's achievement 
of the competencies. 
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Conclusions 

A number of conclusions may be made from an analysis the background 
information in Chapter I and data presented in Chapter m. The following are 
conclusions drawn from this study that are immediately significant to the needs of the 
Texas Higher Education Coordinating Board who sponsored the project: 

1. The assessment model is appropriate for use by The 
Community and Technical College Division of the Texas 
Higher Education Coordinating Board to obtain information 
needed in determining priorities for state programs and state 
leadership activities; 

2. Great progress has been made during the last three years, but 
the degree of implementation of competency-based 
instruction for occupational preparation programs desired 
and mandated by the Coordinating Board is only sixty 
percent; 

3. An understanding of a competency-based instruction and the 
ability to implement a total competency-based instruction 
system by administrators and faculty is the greatest barrier to 
the implementation and successful maintenance of a 
competency-based instruction system. 

4. Program standards or required characteristics of a 
competency-based instruction system have not been 
established and published by the Texas Higher Education 
Coordinating Board; and 

5. The desired degree of the implementation of competency- 
based instruction will be directly dependent on the knowledge 
and skills required for the implementation of CBI being 
developed by administrators and faculty. 

Recommendations 

The following recommendations for increasing the implementation of a 

competency-based instruction in postsecondary occupational technical programs are 
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made for consideration by the staff of Texas Higher Education Coordinating Board, 



members of the Federal Projects Advisory Committee, and members of the Professional 
Development Committee: 



1. Program standards for competency-based instruction be 
established and used by the Texas Higher Education 
Coordinating Board in the evaluation of postsecondary 
occupational/technical programs; 

2. This study should be replicated statewide by specific 
programs over a period of years until all programs have been 
assessed; and 

3. The Texas Higher Education Coordinating Board conduct or 
contract for professional development activities and/or 
programs designed to develop the knowledge and skills 
needed to implement and maintain a competency-based 
instruction system* The professional development activities 
and programs should be conducted until every postsecondary 
administrator and full-time instructor of 
occupational/technical programs has been professionally 
prepared for CBL 
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THE ASSESSMENT OF THE IMPLEMENTATION OF COMPETENCY-BASED INSTRUCTION 

Final Round 



tfeloht Identification of th# reculred characteristice of a oompetency-baaed Instructional program. — 




Characteristics 
Round 3 


Rounds D 




1 Occupational competenciee art obtained from the workplace. 


6.8 I 




2 Academic oompetenoie* needed to devefop and/of perform tha occupational competencies art 
obtained from tha workplace 


5.6 




3 Tha occupational competencies art saquancad for instructional purposes. 


6.9 




4 Tha occupational oompatanciaa art clustarad for course development. 


6.8 




5 Occupational program oouraaa art namad and datcdbad. 


6.0 


6 Tha prerequisite oouraaa ara identified and listed. 


6.0 


7 Tha aupport oouraaa ara Idantiflad and listod. 


6.4 


8 Tachnical and aupport (Including academic courses) ara arranged to that sequential mastery of 
oompatanciaa leads to tha satisfactory performance of ail competencies of tha workplace for the 
occupation (s) being prepared for by tha student. 


6.0 1 




g legitimate or validated career ladder exit points ara identified In the curriculum. 


5.8 1 




10 The ourriculum ia kept current with the requirements of the workplace by the uae of a program 
itfvtaory committee oompoeed of representatives of the workplace. 


6.1 1 


1 1 Occupational competency profit* are developed for the occupational curriculum (program). 


5.0 I 


12 Objectives ara written in performance terme for each occupational competency. 


6,6 s 


13 Objectives are written In performance terme for each occupational task. 


fin 1 


14 Enabling objectives are written for each task performance objective. 


5.5 I 


15 Instructional materials, equipment and supplies essential for performance of the competency are 
available for instructor and student use. 


6.7 I 


16 Learning actMtiea are designed to support the students' ability to learn and perform each 
competency successfully. 


5.7 


17 AH objectivee of the programs (each course, each competency, and each lesson) are written In 
performance terms. 


ft 1 

Q.O 


18 Students entering the occupational program (curriculum) are routinely tested prior to entering the 
\ program for communication, computational, and current occupational ekiUe/aptitudte. 


6.0 


ig Instructional content of the oourse(s) la derived from an analysis of tasks validated in the workplace. 


6.0 


20 The student ie informed of the required competencies and/or tasks prior to Instruction. 


5.9 


21 Learning time le flexible. 


4.7 


22 Learning la guided by feedbeck. 


5.5 


23 Students are informed of performance meaeuree In advance of instruction for each task. 


5.8 


24 Studente ara Informed of performance measures in advance of instruction for each competency. 


5.0 


25 Each student is required to perform each task at a specified standard befors receiving credit or 
I moving on to tha next task. 


5.7 
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28 
30 



Characteristics 
Round 3 



Each student is required to demonstrate mastery of competency at a standard specified by the 
workplace bafora racaiving credit or moving on to tha next competency. 



Rounds 



Students are required to perform each task in a joblike setting. 



A dear specification of student achievement is provided to the student in performance terms prior to 
each learning experience. 



The delivery of instruction or learning experiences provides for Immediate and frequent feedback 
from the student 



The instructor Is a manager and facilitator of learning. 



31 The delivery of instruction Is appropriate for the different learning styles of students. 



32 A record system la maintained for recording and documenting tasks and competencies which have 
been achieved by each student 



5.5 



4.8 



5.3 



5.3 



6.8 



4.0 



6.0 



33 


The record system contains data that includes program standards and performance measures that 
are used in the continuous evaluation of the effectiveness of Instruction In terms of student progress 
and competencies mastered. 


5.9 1 


34 


Task and competency progress records are mamieinwa ioi tmn wuywu. 


6.4 1 


35 


Student progress records are current. 


6.5 1 


36 


Student progress la determined >y criterion-referenced measurement. 


7.0 I 


37 


Competency Profiles are kept current for each student. 


5* 8 


38 


Tha minimum acceptable measure of occupational competency or task achievement is based on the 
penormanoe ww iwuhwivu um wwiij;i»*i 


6.1 1 


39 


A copy of the students 1 competency profile Is Included In the students' permanent records. 


5.9 | 


40 


Students exiting a program are provided with a copy of their competency profile. 


5.9 


41 


Time required for a student to master a competency is not considered In rating or grading a atudent 
K flexible learning time la available. 


4.0 


42 


Each student Is graded on his or her own level of achievement baaed on predetermined standards 
and not In comparison to other students. 


5.8 I 


43 


Gradee for competencies achieved are not lowered by competencies not achieved W flexible learning 
time Is available. 


5.5 1 


44 


The grading system provides for achievement (proficiency) above the established performance 
(minimum) level. 


5.7 B 


45 


Performance levels (criterion) for each occupational competency of the program (course) are 
obtained from the workplace. 


5.9 ~1 


46 


The competency exams are totally performance measured. 


6.4 1 


47 


Student mastery of occupational competencies are determined (assessed) on an Individual basis. 


6.1 1 


48 


Competency exams approved by the workplace are used in determining a student's mastery of an 
occupational competency. 


5.0 I 


49 


Criteria for passing performance tests are acceptable for on-the-job performance. 


5.6 1 


50 


The time taken to complete performance tests Is acceptable for the occupation (workplace). 
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Characteristics 
Round 3 


found* 


51 In an articulattd program or a tach-prap curriculum, both secondary and postsacondary instructors 
havt accaptad tha critaria or standards astablished by tha workplaca to datarmina maatary of aach 
compatancy. 


6.0 


52 Tha sama critaria statad In tha Instructional objactiva ara uaad In tha parformanca tast (axam). 


6,4 I 
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March 10, 1993 



title- first name- last name- 
position?- 
institution- 
address- 
city, state, zip- 

Dear title- last name-: 

The University of North Texas has contracted with the Texas Higher Education 
Coordinating Board to determine the extent to which competency-based instruction is 
being used by public postsecondary institutions to conduct technical/vocational programs. 

We have proposed to determine the use of competency-based instruction by surveying: 
(1) the Deans/Directors of occupational education; (2) Division/Program Chairs/Heads 
of each program and; (3) one instructor of each program This assessment is being made 
to determine the use of competency-based instruction on a statewide basis. Therefore, 
the results of the assessment will not be compiled and reported by individual institution 
unless requested by the respective Dean/Director of the institution. 

Will you assist us in conducting the assessment by distributing to and collecting from 
your faculty the enclosed questionnaires? An addressed postage paid envelope is 
enclosed for your use in returning the completed questionnaires. Please return the 
completed questionnaires on or before 

If we have not included questionnaires for all of your programs, please let me know 
which programs we overlooked. Please call us at (817) 565-4109 if you have any 
questions about the assessment. 

Sincerely, 



Bill E. Lovelace, Director 

Assessment of Competency-Based Instruction 

BEL:pp 
Enclosures 



ERIC 



56 



Appendix B 



Rationale for the Assessment 
of Competency-Based Instruction 

The Texas Higher Education Coordinating Board supports the use of competency- 
based instruction to achieve accountability. The use of a competency-based instruction 
system to deliver a competency-based curriculum is supported by mandate and implied 
mandate. The Terhniml and N ational v™ ? r*m Guidelines (1989) disseminated by the 
Texas Higher Education Coordinating Board provides the following definition of 
competency-based instruction (competency-based education); 

Com petency-based education is designed to teach job related clusters of 
skills and knowledge, the mastery of which forms the basis upon which the 
student is evaluated; competency-based programs must possess the 
following characteristics. 

1. Involvement of business and industry in the 
determination of the job competencies and the 
expected performance level required for 
successful employment within a defined job or 
cluster of jobs. 

2. Course sequence which allows the mastery of 
competencies leading to the satisfactory 
performance of all identified competencies. 

The Texas Higher Education Coordinating Board has contracted with the 

University of North Texas to conduct a statewide assessment to determine the extent to 

which competency-based instruction has been implemented in vocational/technical 

education programs in all Texas public community and technical colleges. 



Appendix B 



Implementation Assessment Questionnaire 
* of Competency-Based Instruction Characteristics 

Deans/Directors Questionnaire 
Part 1 



Instructions: 

This part of the questionnaire is being used to determine the extent to which 
characteristics of a competency-based instructional system are being implemented 01 
your campus for technical/vocational programs. 



1. For each characteristic of competency-based instruction on this part of the 
questionnaire indicate the extent the characteristic has beenimplememed 
for technical/vocational programs on your campus. Circle (x) for "Yes" or 
circle ® for "No." If you circle "N" do not rate the characteristic, go to the 
next characteristic and repeat 1. 

2. If you circled a "Y" for the characteristic then: 

Rate the degree to which you feel that the characteristic has been 
implemented on your campus. Placing a circle around the (l) will indicate 
that the level of implementation of the characteristic is minimal. Placing a 
circle around the © will indicate that level of implementation of the 
characteristic is maximum. 

Go to next characteristic. 



• 



P 



ERIC 



Appendix B 



Implementation Assessment Questionnaire 
of Competency-Based Instruction Characteristics 

Deans/Directors Questionnaire 
Questionnaire 



Characteristics: 
Examples: 




Has This 
Characteristic Been 
Implemented on 
Your Campus? 



Y © 
©N 



Degree of 
Implementation: 
Mingnigp-Mgoman 



1 2 3 4 5 
1 ©3 4 5 
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1 Occupational competencies are obtained from the 
workplace. 

2 Academic competencies needed to develop and/or 
perform the occupational competencies are 
obtained from the workplace. 

3 The occupational competencies are sequenced for 
instructional purposes. 

4 The occupational competencies are clustered for 
course development. 

5 Occupational program courses are named and 
described. 

6 The prerequisite courses are identified and listed. 

7 The support courses are identified and listed. 

8 Technical and support (including academic courses) 
are arranged so that sequential mastery of 
competencies leads to the satisfactory performance 
of all competencies of the workplace for the 
occupation(s) being prepared for by the student. 



Y N 

Y N 

Y N 

Y N 

Y N 

Y N 

Y N 

Y N 



1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4.^ 
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HasJQiis 

Characteristic Been 
Implemented on 

Characteristics: YOUf Campus? 



9 Legitimate or validated career ladder exit points Y N 
are identified in the curriculum. 

10 The curriculum is kept current with the Y N 
requirements of the workplace by the use of a 

program advisory committee composed of 
representatives of the workplace. 

11 Occupational competency profiles are developed Y N 
for the occupational curriculum (program). 



12 Objectives are written in performance terms for 
each occupational competency. 

13 Objectives are written in performance terms for 
each occupational task. 

14 Enabling objectives are written for each task 
performance objective. 



16 Learning activities are designed to support the 
students' ability to learn and perform each 
competency successfully. 

17 All objectives of the programs (each course, each 
competency, and each lesson) are written in 
performance terms. 

♦ 18 Students entering the occupational program 

(curriculum) are routinely tested prior to entering 
the program for communicatiuii, computational, 
and current occupational skills/aptitudes. 

19 Instructional content of the course(s) is derived 
from an analysis of tasks validated in the 
0 workplace. 
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Y N 



Y N 



Y N 



15 Instructional materials, equipment, and supplies Y N 

essential for performance of the competency are 
available for instructor and student use. 



Y N 



Y N 



Y N 



Y N 



Degree of 
Implementation: 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 * 
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Has This 
Characteristic Been 
Implemented on 

Characteristics: Your Campus? 



20 The student is informed of the required Y N 

competencies and/or tasks prior to 
instruction. 



21 Learning time is flexible. 

22 Learning is guided by feedback. 

23 Students are informed of performance measures in 
advance of instruction for each task. 



29 The delivery of instruction or learning experiences 
provides for immediate and frequent feedback from 
the student. 

30 The instructor is a manager and facilitator of 
learning. 

31 The delivery of instruction is appropriate for the 
different learning styles of students. 
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Y N 

Y N 

Y N 



24 Students are informed of performance measures in Y N 
advance of instruction for each competency. 

25 Each student is required to perform each task at a Y N 
specified standard before receiving credit or moving 

on to the next task. 

26 Each student is required to demonstrate mastery of Y N 
competency at a standard specified by the 

workplace before receiving credit or moving on to 
the next competency. 

27 Students are required to perform each task in a Y N 
joblike setting. 

28 A clear specification of student achievement is Y N 
provided to the student in performance terms prior 

to each learning experience. 



Y N 



Y N 



Y N 



Degree of 

Minmum-Maaniiii 

1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 - 
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Characteristics: 



32 A record system is maintained for 

recording and documenting tasks and competencies 
which have been achieved by each student. 

33 The record system contains data that includes 
program standards and performance measures that 
are used in the continuous evaluation of the 
effectiveness of instruction in terms of student 
progress and competencies mastered. 

34 Task and competency progress records are 
maintained for each student. 

35 Student progress records are current. 

36 Student progress is determined by criterion- 
referenced measurement. 

37 Competency Profiles are kept current for each 
student. 

38 The minimum acceptable measure occupational 
competency or task achievement is based on the 
performance level established by the workplace. 

39 A copy of the students* competency profile is 
included in the students' permanent records. 

40 Students exiting a program are provided with a 
copy of their competency profile. 

41 Time required for a student to master a 
competency is not considered in rating or grading a 
student if flexible learning time is available. 

42 Each student is graded on his or her own level of 
achievement based on predetermined standards and 
not in comparison to other students. 



Has This 

Characteristic Been 
Implemented on 
Your Campus? 



Y N 



Y N 



Y N 

Y N 

Y N 

Y N 

Y N 

Y N 

Y N 

Y N 

Y N 



Degree of 



MkomuD^MaMmum 
1 2 3 4 5 

1 2 3 4 5 



1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 
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Characteristics: 



43 Grades for competencies achieved are not lowered 
by competencies not achieved if flexible learning 
time is available. 

44 The grading system provides for achievement 
(proficiency) above the established performance 
(minimum) level. 

45 Performance levels (criterion) for each 
occupational competency of the program (course) 
are obtained from the workplace. 

46 The competency exams are totally performance 
measured. 

47 Student mastery of occupational competencies are 
determined (assessed) on an individual basis. 

48 Competency exams approved by the workplace are 
used in detennining a student's mastery of an 
occupational competency. 

49 Criteria for passing performance tests are 
acceptable for on-the-job performance. 

50 The time taken to complete performance tests is 
acceptable for the occupation (workplace). 

51 In an articulated program or a tech-prep 
curriculum, both secondary and postsecondary 
instructors have accepted the criteria or standards 
established by the workplace to determine mastery 
of each competency. 

52 The same criteria stated in the instructional 
objective are used in the performance test (exam). 



Has This 
Characteristic Been 
Implemented on 
Your Campus? 



Y N 

Y N 

Y N 

Y N 

Y N 

Y N 

Y N 

Y N 

Y N 



Degree of 

I 



\fagniMn-Maxrnirri 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 



Y N 



1 2 3 4 5 
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CBI Assessment Part II Deans/Directors 



BARRIERS TO IMPLEMENTING 
AND/OR MAINTAINING COMPETENCY-BASED INSTRUCTION 

Below is a list of statements that have been considered as barriers to the implementation 
of Competency-Based Instruction. (CBI) 

Please review each statement. If this statement has been a barrier to the 
implementation and/or maintenance of CBI for your program at your institution, please 
place a check under "yes." If this statement has not been a barrier to the 
inplementation and/or maintenance of CBI for your program at your institution, please 
place a check under "no." If you are not sure if the statement has been a barrier to the 
implementation and/or maintenance of CBI for your program, please place a check 
under "don't know." 

Statements 

Organizational Barriers a barrier? 

yes no don t 
know 

0- 1 Faculty are not provided release time needed to — 

develop CBI system format. 

0- 2 These institutions do not have specialists trained in 

CBI systems to assist faculty. 

0- 3 Students are expected to complete occupational — 

preparation programs (or courses) in a specified 
number of hours. 

0- 4 Traditional college procedures do not permit students 

to enter and complete programs with the flexibility 
possible with a CBI system. 

0-5 It is difficult to schedule students and facilities needed ___ 

when using a CBI system. 

0- 6 Students may require greater time to master — 

competencies in a CBI system. 
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Barriers To Implementing a barrier? 

And/Or Maintaining Competency-Based Instruction yes no don't 

know 

0- 7 CBI programs with open entry/open exit do not match 

institutional methods for determining funding. 

0- 8 Faculty have little access to clerical help needed for 

CBI development,implementation, and maintenance* 

0-9 The staff of the Community and Technical College 

Division of the Coordinating Board has not reached 
consensus about Competency-Based Instruction. 

0-10 Guidelines and policies of the Texas Higher Education 

Coordinating Board communicate mixed messages to 
local institutions of the Coordinating Board's 
commitment and direction relative to CBI. 

0-11 There is a lack of a comprehensive State plan and 

methodology for the implementation of CBI. 

0-12 Local administrators do not provide sufficient 

supportive resources needed to encourage faculty 
participation in CBI. 

0-13 Faculty are not aware of opportunities to learn the m 

skills needed to implement CBI. 

0- 14 CBI requires students who are mature and self- 

directed and who can assume responsibility for his/her 
own learning or occupational skills. 

Interpersonal Barriers 

1- 1 Difficulty in getting faculty and institution to accept 

demands of CBI characteristics. 

I- 2 Many inservice, workshops, and conferences are not 

competency-based. 

I- 3 State approved teacher education programs are not 

competency-based. 
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Barriers To Implementing a barrier? 

And/Or Maintaining Competency-Based Instruction yes no don't 



know 



I- 4 CBI system not perceived by faculty as an 

improvement over the existing system for delivery of 
instruction. 

I- 5 Faculty have insufficient knowledge of CBI. 

I- 6 Faculty believe CBI will make new demands on their 
time. 

I- 7 Faculty express a negative attitude about CBI and fail 
to conceptually support its development and/or fail to 
demonstrate and/or commit to goals of CBI. 

I- 8 Adniinistrators express a negative attitude about CBI 
and fail to conceptually support its development 
and/or fail to demonstrate and/or commit to goals of 
CBI. 

I- 9 Faculty have responsibility for CBI program 
development, but lack skills. 

1-10 Teacher education programs do not adequately 
prepare faculty to use a CBI system. 

Ml Faculty fears that the CBI system may result in the 
loss of his/her position. 

Material Barriers 

M- 1 Institutions find it difficult to obtain/maintain 
appropriate equipment required for CBI. 

M- 2 Institutions have difficulty in keeping CBI instructional 
materials current. 

M- 3 There are few, if any, state-provided incentives to 
institutions' administrators and faculty for 
implementing and maintaining a CBI system. 

M- 4 Faculties are not appropriate for CBI. 
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Barriers To Implementing a barrier? 

And/Or Maintaining Competency-Based Instruction yes no don't 

know 

M- 5 Printing, storing, and distributing individualized 

instructional materials require greater resources than 
do conventional programs. 

M- 6 CBI programs require more advanced teaching 

technology that is difficult to obtain and maintain. 

M- 7 State funding for educational instruction is not 

sufficient for CBI programs. 
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March 5, 1993 

Dear Division/Program Chair/Director/Instructional Leader: 

The University of North Texas has contracted with the Texas Higher Education 
Coordinating Board to determine the extent to which competency-based instruction is 
being used by programs by public postsecondary institutic ns to conduct 
technical/vocational instruction. The assessment is being done on a statewide basis. 
Therefore, the results of the assessment will not be reported for each institution. 

Please assist us in conducting the assessment by completing the enclosed questionnaire, 
niacins it the enclosed envelope, and returning it to your Dean or Director for 
occupational education. Your technical Dean/Director has a Glossary of Terms related 
to competency-based instruction. 

If you have any questions about the assessment, call me at (817) 565-4109. 
Sincerely, 



Bill E. Lovelace, Director 

Assessment of Compentency-Based Instruction 

BEL:pp 
Enclosures 
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Rationale for the Assessment 
of Competency-Based Instruction 

The Texas Higher Education Coordinating Board supports the use of competency- 
based instruction to achieve accountability. The use of a competency-based instruction 
system to deliver a competency-based curriculum is supported by mandate and implied 
mandate. The Technical and Vocational Program Guidelines (1989) disseminated by the 
Texas Higher Education Coordinating Board provides the following definition of 
competency-based instruction (competency-based education); 

Competency-based education is designed to teach job related clusters of 
skills and knowledge, the mastery of which forms the basis upon which the 
student is evaluated; competency-based programs must possess the 
following characteristics. 

1. Involvement of business and industry in the 
determination of the job competencies and the 
expected performance level required for 
successful employment within a defined job or 
cluster of jobs. 

2. Course sequence which allows the mastery of 
competencies leading to the satisfactory 
performance of all identified competencies. 

The Texas Higher Education Coordinating Board has contracted with the 

University of North Texas to conduct a statewide assessment to determine the extent to 

which competency-based instruction has been implemented in vocational/technical 

education programs in all Texas public community and technical colleges. 
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Program Area: area- 
CIP Division : division- 



Implementation Assessment Questionnaire 
of Competency-Based Instruction Characteristics 

Division/Program Chair/Director 
Part 1 



Instructions: 

This part of the questionnaire is being used to determine the extent to which 
characteristics of a competency-based instructional system are being implemented on 
your campus for the (name of program area) program area. 

1. For each characteristic of competency-based instruction on this part of the 
questionnaire indicate the extent the characteristic has been implemented 
for technical/vocational programs on your campus. Circle (£) for "Yes" or 
circle ® for "No." If you circle "N" do not rate the characteristic, go to the 
next characteristic and repeat 1. 

2. If you circled a "Y" for the characteristic then: 

Rate the degree to which you feel that the characteristic has been 
implemented on your campus. Placing a circle around the (T) will indicate 
that the level of implementation of the characteristic is minimal. Placing a 
circle around the (D will indicate that level of implementation of the 
characteristic is maximum. 

Go to next characteristic. 
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Program Area: area- 
CIP Division : division- 



Implementation Assessment Questionnaire 
of Competency-Based Instruction Characteristics 

Division/Program Chair/Director 
Questionnaire 
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Characteristics: 
Examples: 

1. IIIIHI 

2. iiiii! 

1 Occupational competencies are obtained from the 
workplace. 

2 Academic competencies needed to develop and/or 
perform the occupational competencies are 
obtained from the workplace. 

3 The occupational competencies are sequenced for 
instructional purposes. 

4 The occupational competencies are clustered for 
course development. 

5 Occupational program courses are named and 
described. 

6 The prerequisite courses are identified and listed. 

7 The support courses are identified and listed. 

8 Technical and support (including academic courses) 
are arranged so that sequential mastery of 
competencies leads to the satisfactory performance 
of all competencies of the workplace for the 
occupation(s) being prepared for by the student. 



Has This 

Characteristic Been 
Implemented on 
Your Campus? 



Y © 
© N 

Y N 

Y N 

Y N 

Y N 

Y N 



Y 
Y 
Y 



N 
N 
N 



De gree of 
Implementation: 

Mkinum-Maxiaium 



1 2 3 4 5 

1 ©3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 
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Characteristics; 



Has This 

Characteristic Been 
Implemented on 
Your Campus? 



De gree of 
Implementation: 

Mkumun>^viaxrnurn 



9 Legitimate or validated career ladder exit points 
are identified in the curriculum. 

10 The curriculum is kept current with the 
requirements of the workplace by the use of a 
program advisory committee composed of 
representatives of the workplace. 

11 Occupational competency profiles are developed 
for the occupational curriculum (program). 

12 Objectives are written in performance terms for 
each occupational competency. 

13 Objectives are written in performance terms for 
each occupational task. 

14 Enabling objectives are written for each task 
performance objective. 

15 Instructional materials, equipment, and supplies 
essential for performance of the competency are 
available for instructor and student use. 

16 Learning activities are designed to support the 
students' ability to learn and perform each 
competency successfully. 

17 All objectives of the programs (each course, each 
competency, and each lesson) are written in 
performance terms. 

18 Students entering the occupational program 
(curriculum) are routinely tested prior to entering 
the program for communication, computational, 
and current occupational skills/aptitudes. 

19 Instructional content of the course(s) is derived 
from an analysis of tasks validated in the 
workplace. 



Y N 



Y N 



Y N 



Y N 



Y N 



Y N 



Y N 



Y N 



Y N 



Y N 



Y N 



1 2 3 4 5 



1 2 3 4 5 



1 2 3 4 5 



1 2 3 4 5 



1 2 3 4 5 



1 2 3 4 5 



1 2 3 4 5 



1 2 3 4 5 



1 2 3 4 5 



1 2 3 4 5 



1 2 3 4 5 
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Has This 

Characteristic Been 
Implemented on 

Characteristics: Your Campus? 
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20 The student is informed of the required Y N 
competencies and/or tasks prior to 

instruction. 

21 Learning time is flexible. Y N 

22 Learning is guided by feedback, Y N 

23 Students are informed of performance measures in Y N 
advance of instruction for each task. 

24 Students are informed of performance measures in Y N 
advance of instruction for each competency. 

25 Each student is required to perform each task at a Y N 
specified standard before receiving credit or moving 

on to the next task. 

26 Each student is required to demonstrate mastery of Y N 
competency at a standard specified by the 

workplace before receiving credit or moving on to 
the next competency. 

27 Students are required to perform each task in a Y N 
joblike setting. 

28 A clear specification of student achievement is Y N 
provided to the student in performance terms prior 

to each learning experience. 

29 The delivery of instruction jr learning experiences Y N 
provides for immediate and frequent feedback from 

the student. 

30 The instructor is a manager and facilitator of Y N 
learning. 

31 The delivery of instruction is appropriate for the Y N 
different learning styles of students. 

73 



Degree of 
Implementation: 

Miosnum-Maxknun 

1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 
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Characteristics: 



32 A record system is maintained for 

recording and documenting tasks and competencies 
which have been achieved by each student. 

33 The record system contains data that includes 
program standards and performance measures that 
are used in the continuous evaluation of the 
effectiveness of instruction in terms of student 
progress and competencies mastered. 

34 Task and competency progress records are 
maintained for each student. 

35 Student progress records are current. 

36 Student progress is determined by criterion- 
referenced measurement. 



Has This 

Characteristic Been 
Implemented on 
Your Campus? 



Y N 



Y N 



Y N 

Y N 

Y N 



Degree of 
Implementation: 
Minknum-Marirnum 

1 2 3 4 5 



1 2 3 4 5 



1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 



37 Competency Profiles are kept current for each 
student. 



Y N 



1 2 3 4 5 



38 The minimum acceptable measure occupational 
competency or task achievement is based on the 
performance level established by the workplace. 

39 A copy of the students' competency profile is 
included in the students' permanent records. 

40 Students exiting a program are provided with a 
copy of their competency profile. 

41 Time required for a student to master a 
competency is not considered in rating or grading a 
student if flexible learning time is available. 

42 Each student is graded on his or her own level of 
achievement based on predetermined standards and 
not in comparison to other students. 



Y N 



Y N 



Y N 



Y N 



Y N 



1 2 3 4 5 



1 2 3 4 5 



1 2 3 4 5 



1 2 3 4 5 



1 2 3 4 5 
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Characteristics: 



Has This 

Characteristic Been 
Implemented on 
Your Campus? 



Degree of 
Implementation: 
Minimum-Maxkmiii 



43 Grades for competencies achieved are not lowered 
by competencies not achieved if flexible learning 
time is available. 



Y N 



1 2 3 4 5 



44 The grading system provides for achievement 
(proficiency) above the established performance 
(minimum) level. 

45 Performance levels (criterion) for each 
occupational competency of the program (course) 
are obtained from the workplace. 

46 The competency exams are totally performance 
measured. 



Y N 



Y N 



Y N 



1 2 3 4 5 



1 2 3 4 5 



1 2 3 4 5 



47 Student mastery of occupational competencies are 
determined (assessed) on an individual basis. 

48 Competency exams approved by the workplace are 
used in determining a student's mastery of an 
occupational competency. 

49 Criteria for passing performance tests are 
acceptable for on-the-job performance. 

50 The time taken to complete performance tests is 
acceptable for the occupation (workplace). 

51 In an articulated program or a tech-prep 
curriculum, both secondary and postsecondary 
instructors have accepted the criteria or standards 
established by the workplace to determine mastery 
of each competency. 

52 The same criteria stated in the instructional 
objective are used in the performance test (exam). 



Y N 



Y N 



Y N 



Y N 



Y N 



1 2 3 4 5 



1 2 3 4 5 



1 2 3 4 5 



1 2 3 4 5 



1 2 3 4 5 



Y N 



1 2 3 4 5 
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CBI Assessment Part II Division/Program 
Program Area: (to be typed in) Chairs/Directors 
CIP Division : (to be typed in) 



BARRIERS TO IMPLEMENTING 
AND/OR MAINTAINING COMPETENCY-BASED INSTRUCTION 

Below is a list of statements that have been considered as barriers to the implementation 
of Competency-Based Instruction. (CBI) 

Please review each statement. If this statement has been a barrier to the 
implementation and/or maintenance of CBI for your program at your institution, please 
place a check under "yes." If this statement has not been a barrier to the 
inplementation and/or maintenance of CBI for your program at your institution, please 
place a check under "no." If you are not sure if the statement has been a barrier to the 
implementation and/or maintenance of CBI for your program, please place a check 
under "don't know" 

Statements 

Or ganizational Barrie rs a barrier? 

yes no don't 
know 

0- 1 Faculty are not provided release time needed to 

develop CBI system format. 

0- 2 These institutions do not have specialists trained in 

CBI systems to assist faculty. 

0- 3 Students are expected to complete occupational 

preparation programs (or courses) in a specified 
number of hours. 

0- 4 Traditional college procedures do not permit students 

to enter and complete programs with the flexibility 
possible with a CBI system. 

0- 5 It is difficult to schedule students and facilities needed 

when using a CBI system. 

0- 6 Students may require greater time to master 

competencies in a CBI system. 
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Barriers To Implementing a barrier? 

And /Or Maintaining Competency-Based Instruction yes no don't 

know 

0- 7 CBI programs with open entry/open exit do not match 

institutional methods for determining funding. 



0- 8 Faculty have little access to clerical help needed for 
CBI development,implementation, and maintenance. 

0- 9 The staff of the Community and Technical College 
Division of the Coordinating Board has not reached 
consensus about Competency-Based Instruction. 

0-10 Guidelines and policies of the Texas Higher Education 
Coordinating Board communicate mixed messages to 
local institutions of the Coordinating Board's 
commitment and direction relative to CBI. 

0-11 There is a lack of a comprehensive State plan and 
methodology for the implementation of CBI. 

0-12 Local administrators do not provide sufficient 

supportive resources needed to encourage faculty 
participation in CBI. 

0-13 Faculty are not aware of opportunities to learn the 
skills needed to implement CBI. 

0- 14 CBI requires students who are mature and self- 

directed and who can assume responsibility for his/her 
own learning or occupational skills. 

Interpersonal Barriers 

1- 1 Difficulty in getting faculty and institution to accept 

demands of CBI characteristics. 



I- 2 Many inservice, workshops, and conferences are not 
competency-based. 

I- 3 State approved teacher education programs are not 
competency-based. 
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Barriers To Implementing a barrier? 

And/Or Maintaining Competency-Based Instruction yes no don't 



know 



I- 4 CBI system not perceived by faculty as an 

improvement over the existing system for delivery of 
instruction. 

I- 5 Faculty have insufficient knowledge of CBI. 

I- 6 Faculty believe CBI will make new demands on their 
time. 

I- 7 Faculty express a negative attitude about CBI and fail 
to conceptually support its development and/or fail to 
demonstrate and/or commit to goals of CBI. 

I- 8 Administrators express a negative attitude about CBI 
and fail to conceptually support its development 
and/or fail to demonstrate and/or commit to goals of 
CBI. 

I- 9 Faculty have responsibility for CBI program 
development, but lack skills. 

MO Teacher education programs do not adequately 
prepare faculty to use a CBI system. 

Ml Faculty fears that the CBI system may result in the 
loss of his/her position. 

Material Barriers 

M- 1 Institutions find it difficult to obtain/maintain 
appropriate equipment required for CBI. 

M- 2 Institutions have difficulty in keeping CBI instructional 
materials current. 

M- 3 There are few, if any, state-provided incentives to 
institutions' administrators rnd faculty for 
implementing and maintaining a CBI system. 

M- 4 Facilities are not appropriate for CBI. 
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Barriers To Implementing 
And/Or Maintaining Competency-Based Instruction 



a barrier? 
yes no don't 
know 



M- 5 Printing, storing, and distributing individualized 

instractional materials require greater resources than 
do conventional programs. 

M- 6 CBI programs require more advanced teaching 

technology that is difficult to obtain and maintain. 

M- 7 State funding for educational instruction is not 
sufficient for CBI programs. 
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March 5, 1993 
Dear Instructor: 

The University of North Texas has contracted with the Texas Higher Education 
Coordinating Board to determine the extent to which competency-based instruction is 
being used by programs by public postsecondary institutions to conduct 
technical/vocational instruction. The assessment is being done on a statewide basis. 
Therefore, the results of the assessment will not be reported for each institution. 

Please assist us in conducting the assessment by completing the enclosed questionnaire, 
placing it the enclosed envelope, and returning it to your Dean or Director for 
occupational education. Your technical Dean/Director has a Glossary of Terms related 
to competency-based instruction. 

If you have any questions about the assessment, call me at (817) 565-4109. 
Sincerely, 



Bill E, Lovelace, Director 

Assessment of Compentency-Based Instruction 



BEL:pp 
Enclosures 
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Rationale for the Assessment 
of Competency-Based Instruction 

The Texas Higher Education Coordinating Beard supports the use of competency- 
based instruction to achieve accountability. The use of a competency-based instruction 
system to deliver a competency-based curriculum is supported by mandate and implied 
mandate. The Technical anH Vocational Program Guidelines (1989) disseminated by the 
Texas Higher Education Coordinating Board provides the following definition of 
competency-based instruction (competency-based education); 

Competency-based education is designed to teach job related clusters of 
skills and knowledge, the mastery of which forms the basis upon which the 
student is evaluated; competency-based programs must possess the 
following characteristics. 

1. Involvement of business and industry in the 
determination of the job competencies and the 
expected performance level required for 
successful employment within a defined job or 
cluster of jobs. 

2. Course sequence which allows the mastery of 
competencies leading to the satisfactory 
performance of all identified competencies. 

The Texas Higher Education Coordinating Board has contracted with the 

University of North Texas to conduct a statewide assessment to determine the extent to 

which competency-based instruction has been implemented in vocational/technical 

education programs in all Texas public community and technical colleges. 
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Program Area: area- Name of Program CiP 

Division : division- 

(Please write in name of program 

that you teach.) 



Implementation Assessment Questionnaire 
of Competency-Based Instruction Characteristics 

Instructors 
Part 1 



Instructions: 

This part of the questionnaire is being used to determine the extent to which 
characteristics of a competency-based instructional system are being implemented on 
your campus for the area2~ program area. 



1. For each characteristic of competency-based instruction on this part of the 
questionnaire indicate the extent the characteristic has beeriimplemented 
for technical/vocational programs on your campus. Circle ® for "Yes" or 
circle ® for "No." If you circle "N" do not rate the characteristic, go to the 
next characteristic and repeat 1. 

2. If you circled a T for the characteristic then: 

Rate the degree to which you feel that the characteristic hasbeen 
implemented on your campus. Placing a circle around the (D will indicate 
that the level of implementation of the characteristic is minimal. Placing a 
circle around the © will indicate that level of implementation of the 
characteristic is maximum. 

Go to next characteristic. 
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Program Area: area- 
CIP Division : division- 

Implementation Assessment Questionnaire 
of Competency-Based Instruction Characteristics 

Division/Program Chair/Director 
Questionnaire 



Characteristics; 
Examples: 

1- 



1 Occupational competencies are obtained from the 
worlq lace. 

2 Academic competencies needed to develop and/or 
perform the occupational competencies are 
obtained from the workplace. 

3 The occupational competencies are sequenced for 
instructional purposes. 

4 The occupational competencies are clustered for 
course development. 

5 Occupational program courses are named and 
described. 

6 The prerequisite courses are identified and listed. 

7 The support courses are identified and listed. 

8 Technical and support (including academic courses) 
are arranged so that sequential mastery of 
competencies kads to the satisfactory performance 
of all competencies of the workplace for the 
occupation(s) being prepared for by the student. 



Has This 

Characteristic Been 
Implemented on 
Your Campus? 



Y © 
® N 

Y N 

Y N 

Y N 

Y N 

Y N 

Y N 

Y N 

Y N 



Degree of 
Implementation: 

MbimunvMaamum 



1 2 3 4 5 

1 ©3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 



9 
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Characteristics: 



9 Legitimate or validated career ladder exit points 
are identified in the curriculum. 

10 The curriculum is kept current with the 
requirements of the workplace by the use of a 
program advisory committee composed of 
representatives of the workplace. 

11 Occupational competency profiles are developed 
for the occupational curriculum (program). 

12 Objectives are written in performance terms for 
each occupational competency. 

13 Objectives are written in performance terms for 
each occupational task. 

14 Enabling objectives are written for each task 
performance objective. 

15 Instructional materials, equipment, and supplies 
essential for performance of the competency are 
available for instructor and student use. 

16 Learning activities are designed to support the 
students' ability to learn and perform each 
competency successfully. 

17 All objectives of the programs (each course, each 
competency, and each lesson) are written in 
performance terms. 

18 Students entering the occupational program 
(curriculum) are routinely tested prior to entering 
the program for communication, computational, 
and current occupational skills/aptitudrs. 

19 Instructional content of the course(s) is derived 
from an analysis of tasks validated in the 

q workplace. 

ERIC 



Has This 

Characteristic Been 
Tmplemented on 
Ypnr Campus? 



Y N 



Y N 



Y N 

Y N 

Y N 

Y N 

Y N 

Y N 

Y N 

Y N 



Y N 



Degree of 
Implementation: 

Mmimum-MsdmuTi 
1 2 3 4 5 



1 2 3 4 5 



1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 
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Has This 

Characteristic Been 
Im plemented on 

Characteristics: Ypur Campus? 



20 The student is informed of the required Y N 
competencies and/or tasks prior to 

instruction. 

21 Learning time is flexible. Y N 



22 Learning is guided by feedback. 

23 Students are informed of performance measures in 
advance of instruction for each task. 



27 Students are required to perform each task in a 
joblike setting. 

28 A clear specification of student achievement is 
provided to the student in performance terms prior 
to each learning experience. 



31 The delivery of instruction is appropriate for the 
different learning styles of students. 
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Y N 

Y N 



24 Students are informed of performance measures in Y N 
advance of instruction for each competency. 

25 Each student is required to perform each task at a Y N 
specified standard before receiving credit or moving 

on to the next task. 

26 Each student is r quired to demonstrate mastery of Y N 
competency at l standard specified by the 

workplace before receiving credit or moving on to 
the next competency. 



Y N 



Y N 



29 The delivery of instruction or learning experiences Y N 
provides for immediate and frequent feedback from 

the student. 

30 The instructor is a manager and facilitator of Y N 
learning. 



Y N 



v. rr 



D egree of 
implementation: 

1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 
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Characteristics: 



32 A record system is maintained for 

recording and documenting tasks and competencies 
which have been achieved by each student. 

33 The record system contains data that includes 
program standards and performance measures that 
are used in the continuous evaluation of the 
effectiveness of instruction in terms of student 
progress and competencies mastered. 

34 Task and competency progress records are 
maintained for each student. 

35 Student progress records are current. 

36 Student progress is determined by criterion- 
referenced measurement. 



Has This 

Characteristic Been 
Implemented on 
Your Campus? 



Y N 



Y N 



Y N 

Y N 

Y N 



Degree of 
Implementation: 

Nlinimun>Maxrriurn 

1 2 3 4 5 



1 2 3 4 5 



1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 



37 Competency Profiles are kept current for each 
indent. 

38 The minimum acceptable measure occupational 
competency or task achievement is based on the 
performance level established by the workplace. 

39 A copy of the students' competency profile is 
included in the students' permanent records. 

40 Students exiting a program are provided with a 
copy of their competency profile. 

41 Time required for a student to master a 
competency is not considered in rating or grading a 
student if flexible learning time is available. 

42 Each student is graded on his or her own level of 
achievement based on predetermined standards and 
not in comparison to other students. 



Y N 



Y N 



Y N 



Y N 



Y N 



Y N 



1 2 3 4 5 



1 2 3 4 5 



1 2 3 4 5 



1 2 3 4 5 



1 2 3 4 5 



1 2 3 4 5 
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Characteristics: 



43 Grades for competencies achieved are not lowered 
by competencies not achieved if flexible learning 
time is available. 

44 The grading system provides for achievement 
(proficiency) above the established performance 
(minimum) level. 

45 Performance levels (criterion) for each 
occupational competency of the program (course) 
are obtained from the workplace. 

46 The competency exams are totally performance 
measured. 

47 Student mastery of occupational competencies are 
determined (assessed) on an individual basis. 

48 Competency exams approved by the workplace are 
used in determining a students mastery of an 
occupational competency. 

49 Criteria for passing performance tests are 
acceptable for on-the-job performance. 

50 The time taken to complete performance tests is 
acceptable for the occupation (workplace). 

51 In an articulated program or a tech-prep 
curriculum, both secondary and postsecondary 
instructors have accepted the criteria or standards 
established by the workplace, to determine mastery 
of each competency. 

52 The same criteria stated in the instructional 
objective are used in the performance test (exam). 



Has This 

Characteristic Beep 
Implemented pn 
Your Campus? 



Y N 

Y N 

Y N 

Y N 

Y N 

Y N 

Y N 

Y N 

Y N 



Degree of 
Implementation; 

Miramum-Mawrauni 
1 2 3 4 5 



1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 



Y N 



1 2 3 4 s 



9 
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CBI Assessment Part II Name of Program 

Program Area: (to be typed in) 

CIP Division : (to be typed in) (Please write in name of program 

that you teach.) 

BARRIERS TO IMPLEMENTING 
AND/OR MAINTAINING COMPETENCY-BASED INSTRUCTION 

Below is a list of statements that have been considered as barriers to the implementation 
of Competency-Based Instruction. (CBI) 

Please review each statement. If this statement has been a barrier to the 
implementation and/or maintenance of CBI for your program at your institution, please 
place a check under "yes." If this statement has not been a barrier to the 
implementation and/or maintenance of CBI for your program at your institution, please 
place a check under "no." If you are not sure if the statement has been a barrier to the 
implementation and/or maintenance of CBI for your program, please place a check 
under "don't know." 

Statements 

Or ganizational Barriers a barrier? 

yes no don't 
know 

0- 1 Faculty are not provided release time needed to 

develop CBI system format. ~ ~~ ~~ 

0- 2 These institutions do not have specialists trained in 

CBI systems to assist faculty. ~~ ~ 

0- 3 Students are expected to complete occupational 

preparation programs (or courses) in a specified 
number of hours* 

0- 4 Traditional college procedures do not permit students 

to enter and complete programs with the flexibility ~~ 
possible with a CBI system. 

4 0- 5 It is difficult to schedule students and facilities needed 

when using a CBI system. 

0- 6 Students may require greater time to master 

competencies in a CBI system. 

er|c b8 
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Barriers To Implementing a barrier? 

And/Or Maintaining Competency-Based Instruction yes no don't 



know 



0- 7 CBI programs with open entry/open exit do not match 
institutional methods for determining funding. 

0- 8 Faculty have little access to clerical help needed for 
CBI development, implementation, and maintenance. 

0- 9 The staff of the Community and Technical College 
Division of the Coordinating Board has not reached 
consensus about Competency-Based Instruction. 

0-10 Guidelines and policies of the Texas Higher Education 
Coordinating Board communicate mixed messages to 
local institutions of the Coordinating Board's 
commitment and direction relative to CBI. 

0-11 There is a lack of a comprehensive State plan and 
methodology for the implementation of CBI. 

0-12 Local administrators do not provide sufficient 

supportive resources needed to encourage faculty 
participation in CBI. 

0-13 Faculty are not aware of opportunities to learn the 
skills needed to implement CBI. 

0- 14 CBI requires students who are mature and self- 

directed and who can assume responsibility for his/her 
own learning or occupational skills. 

Interpersonal Barriers 

1- 1 Difficulty in getting faculty and institution to accept 

demands of CBI characteristics. 

I- 2 Many inservice, workshops, and conferences are not 
competency-based. 

I- 3 State approved teacher education programs are not 
competency-based. 
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Barriers To Implementing 
And/Or Maintaining Competency-Based Instruction 



a harrier? 
yes no don't 
know 



1- 4 CBI system not perceived by faculty as an 

improvement over the existing system for delivery of 
instruction. 

I- 5 Faculty have insufficient knowledge of CBI. 

I- 6 Faculty believe CBI will make new demands on their 
time. 

I- 7 Faculty express a negative attitude about CBI and fail 
to conceptually support its development and/or fail to 
demonstrate and/or commit to goals of CBI. 

I- 8 Administrators express a negative attitude about CBI 
and fail to conceptually support its development 
and/or fail to demonstrate and/or commit to goals of 
CBI. 

I- 9 Faculty have responsibility for CBI program 
development, but lack skills. 

1-10 Teacher education programs do not adequately 
prepare faculty to use a CBI system. 

Ml Faculty fears that the CBI system may result in the 
loss of his/her position. 

Material Barriers 

M- 1 Institutions find it difficult to obtain/maintain 
appropriate equipment required for CBI. 

Ni- 2 Institutions have difficulty in keeping CBI instructional 
materials current. 

M- 3 There are few, if any, state-provided incentives to 
instil :tions' administrators and faculty for 
implementing and maintaining a CBI system. 

M- 4 Facilities are not appropriate for CBI. 
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Barriers To Implementing a barrier? 

And/Or Maintaining Competency-Based Instruction yes no don't 

know 

M- 5 Printing, storing, and distributing individualized 

instructional materials require greater resources than 
do conventional programs. 

M- 6 CBI programs require more advanced teaching 

technology that is difficult to obtain and maintain. 

M- 7 State funding for educational instruction is not 

sufficient for CBI programs. 
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Texas Public Community and Technical Colleges 
Participating in the Assessment of Competency-Based Instruction 

Alvin Community College 
Amarillo College 
Angelina College 

Austin Community College 

Bee County College 

Blimi College 

Brazosport College 

Central Texas College 

Cisco Junior College 

Cooke County College 

Collin County Community College 

North Lake College 

Richland College 

El Paso Community College 

Frank Phillips Community College 

Galveston College 

Hill College 

Houston Community College 
Howard College at Big Spring 
Kilgore College 

Lamar Institution of Technology at Beaumont 
Lamar University at Orange 
Lamar University at Port Arthur 
Laredo Junior College 
Lee College 

iMcLennan Community College 
Midland College 

Northeast Texas Community College 

Northlake College 

Odessa College 

Palo Alto College 

Panola College 

Paris Junior College 

Ranger Junior College 

San Antonio College 

San Jacinto College-Central Campus 

San Jacinto College-North Campus 

San Jacinto College-South Campus 

South Plains College 

St. Philip's College 
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Texas Southmost College-U.T. Brownsville 

Texas State Technical College-Amarillo 

Texas State Technical College-Harlingen 

Texas State Technical College-Sweetwater 

Trinity Valley Community College 

Tyler Junior College 

Vernon Regional Junior College 

The Victoria College 

Weatherford College 

Western Texas College 



APPENDIX D 

A Model for Evaluating 
Competency-Based Instruction 
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A MODEL FOR EVALUATING 
COMPETENCY-BASED INSTRUCTION 



University of North Texas 



June 1993 



A MODEL FOR EVALUATING 
COMPETENCY - BASED INSTRUCTION 



Prepared for 
The Texas Higher Education Coordinating Board 

by 

Bill E. Lovelace 



School of Merch Rising and Hospitality Management 
University of North Texas 



June 1993 



I. Introduction 



During the past forty years the national cost of education has increased at a rate 
three times as fast as the Gross National Product. With educational and training 
enrollment increasing more rapidly than the population, the cost per student is increasing 
faster than the appraised valui of taxable property. In addition to the present trends of 
education for all three has been a significant effort made to recover a large population 
of individuals who have dropped out of the educational process at an early age and those 
individuals who must be upgraded in order maintain their present position in the 
workplace. Inese trends lead to rapidly increasing taxes for education at the local, state, 
and federal level. 

With growing demand for additional resources for education and taxpayers 
reluctance to provide more funds brings about the need for accountability. Educators 
can no longer afford to make plans and set objectives without taking into consideration 
the need for accountability to demonstrate to the funding source that the funding source 
is receiving a quality product for the funds provided. 

Efforts have been made at the local, state, and national level to design performance 
accountability systems. In the summer of 1987, the U.S. Department of Education 
initiated a series of stuu ; es and work groups to address the problem of accountability in 
education. The creation of responsible and responsive accountability systems was 
assigned to the Office of Educational Research and Improvement (OERI) of the 

♦ 

U.S.D.O.E., by the Secretary of Education. A study group was created to address the 
problem of accountability. The Report of the OERI State Accountability Study Group 
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was completed in the fall of 1988. 

The study group not only reviewed the need and process for designing an 
accountability system, they also studied the role of state and federal agencies in 
providing oversight or technical assistance in developing and implementing accountability 
systems. The OERI reports stated: "Inherent in the concept of accountability is the 
notion of oversight — monitoring to ensure that public monies are spent in ways that 
produce acceptable levels of education performance." Monitoring, as used in relation to 
accountability, is different than the use of monitoring for compliance. Monitoring or 
oversight for educational programs or projects conducted under grant or contract has 
included technical assistance to the local project manager by the federal project program 
officer or by a state staff advisor. 

Past research and experience clearly demonstrates the need for focused technical 
assistance. It has been determined that the relative effectiveness of different forms of 
assistance, when tied to accountability **nd performance, is still largely unknown. It was 
concluded by the OERI study group that: "If state recognition of performance rests 
upon an application process, the state agency should offer technical assistance...." This 
would be especially true when a local community college has set its own goals and 
performance measures for the delivery of a specified product or service. 

The Perkins Act (PL 101-392) mandates accountability by requiring recipients of 
Title II C funds to evaluate themselves on an annual basis using performance measures 
and standards established by the State Board for Vocational Education. If the results of 
the local evaluation indicates that local programs are not making sufficient progress in 



meeting the standard established by the State, the local recipients of Perkins funds are 
required to develop and implement a plan for the improvement of the program(s). 

The state agency responsible for the administration of Perkins funds also has an 
implied responsibility of accountability for State programs and leadership activities 
designed to improve local programs on a statewide basis. This accountability can be 
achieved by formative evaluation or assessment. 

The Texas Higher Education Coordinating Board supports the use of competency- 
based instruction to achieve accountability. The use of a competency-based instruction 
system to deliver a competency-based curriculum is supported by mandate and implied 
mandate. The Technical and Vocational P rogram Guidelines disseminated by the Texas 
Higher Education Coordinating Board provides the following definition of competency- 
based instruction (competency-based education); 

Competency-based education is designed to teach job related 
clusters of skills and knowledge, the mastery of which forms 
the basis upon which the student is evaluated; competency- 
based programs must possess the following characteristics. 

1. Involvement of business and industry in the 
determination of the job competencies and the 
expected performance level required for 
successful employment within a defined job or 
cluster of jobs. 

2. Course sequence which allows the mastery of 
competencies leading to the satisfactory 
performance of all identified competencies. 
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II. The Evaluation Process 



Evaluation is the collection and reporting of data that can be used in making 
decisions for planning and determining accountability. Evaluation is a process that is 
designed to provide information in a formative or summative mode for management 
decisions. The primary focus of formative evaluation (assessment) in education is to 
determine for planning purposes the status of an educational system or a specific 
component or process within an educational system. 

Formative evaluation or assessment is normally done by collecting data that are 
quantitative. The assessment is conducted to gather specified information about status 
and comparing the findings or analyzed status information with predetermined goals, 
objectives, or characteristics program, component, or process being assessed. 

An assessment program or project must include a process for "discrepancy analysis" 
which will pinpoint the specific purpose for which the assessment is made. The analysis 
of information collected by the formative evaluation or assessment must be able to 
identify "what is" cn^i compare the "what is" to the "what should be" or desired status 
that is described by characteristics or measures of prestated goals and objectives. 

In the development and implementation of an assessment activity or program, the 
planners must define their terms very carefully and specify the purposes of the 
assessment to be conducted. Before any assessment activities are planned, there are two 
important assumptions that should be considered. 

One assumption that can be made is that the planned assessment can obtain 
information that is needed and has not been compiled ar.d is not readily available 
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elsewhere. Therefore, the question that becomes important for a specific assessment 
project is: "Has a methodology been developed and proven to produce the specific 
information that the proposed assessment project is expected to obtain?" 

The second assumption which can be made is that the information obtained by the 
assessment can and will be used by the decision makers to improve the educational 
component or process being assessed. The ultimate design of any assessment is 
dependent upon the specific purpose of the assessment. 

Summative evaluation or product evaluation is the best type of evaluation to 
determine accountability of an occupational preparation program. The types of 
information collected by product or summative evaluation include: 

• student outcomes or achievements as they relate to program 
standards and objectives 

• measures of student progress and outcomes as they relate 
program standards and instructional objectives 

Information collected and analyzed from product evaluation is used primarily to 

determine if a program should be continued, terminated, or modified. Since one of the 

decisions that can be made from product evaluation is "modify" the program, process 

.. — 

evaluation information should be included in the summative or product evaluation plan. 

The requirement for an annual evaluation at the local level by Perkins implies that 
the evaluation is to be continuous. Therefore, information on measures of student 
progress will be as important to the local annual evaluation as information on program 
standards or student outcomes will be in determining program accountability. 
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The need for including process information in the evaluation plan is emphasized by 



Alan Ginsburg, who stated: 

One criticism was that evaluators showed a preoccupation with 
measuring overall program impacts, particularly test score changes, 
while achievement outcomes are important, they don't tell the whole 
story. "Black box" evaluations that ignore program processes are 
particularly frustrating in that, by themselves, they fail to indicate how 
to improve poorly performing programs. 1 



Ginsburg, Alan et al, Reinvigorating Program Evaluation at 
the U.S. Department of Education, EDUCATIONAL RESEARCHER, Vol.21, 
Number 3, April 1992 
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EL Evaluation Models 



The RFA for this project only required the development of a statewide evaluation 
process or model for determining the extent to which competency-based instruction had 
been implemented in all public community and technical colleges in Texas, The 
statewide evaluation process (model) or assessment was developed for the purposes of 
collecting information that could be used by the staff of the Texas Higher Education 
Coordinating Board in determining what state leadership activities or technical assistance 
should be provided to the community and technical colleges for a statewide 
implementation of a competency-based instruction system. 

Concurrent with need for a statewide assessment process there is also a need for an 
evaluation process or model that can be used at the local institution level to determine 
the effectiveness of competency-based instruction. The contractor, University of North 
Texas, proposed, in addition to the state assessment process, to develop a conceptual 
evaluation model to be used at the institutional level. 

The components of the two models are described below: 



Statewide Assessment 



Local Evaluation 



Establish purpose of assessment 



Establish purpose of evaluation 



Assessment Committee 



Evaluation steering committee 



Establish standards for CBI 



Select evaluation criteria 



Establish assessment criteria 



On-site visits 



Develop and mail questionnaires to 
administrators and faculty 



Steering committee meetings 
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Compile and analyze responses 

Perform discrepancy analysis 

Identify strategies and resources to 
be provided by the state to local 
institutions for the improvement of 
CBI 



Conduct interviews with students, 
faculty, and administrators 

Survey students, faculty, and 
employers 

Analyze findings 

Report results of evaluation with 
recommendations for improvement 
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Model for Statewide Assessment 



1. 



Establish purpose of the assessment 



The purpose of this assessment is to determine the extent to which competency- 
based instruction in technical/vocational education programs have been 
implemented in all public community and technical colleges. 

2. Assessment Committee 

For the purpose of the statewide assessment, a project advisory committee was 
created and is being used. 

3. Establish Standards for CBI 

Any evaluation must be done in relation to stated goals, objectives, or standards. 
Since goals, objectives, or standards for competency-based instruction had not 
been established by the Texas Higher Education Coordinating Board, it became 
the responsibility of the evaluator to establish characteristics for which evaluative 
criteria could be developed. The evaluator, the University of North Texas did not 
feel that it had the prerogative of establishing standards for the State. Therefore, 
it was decided to establish characteristics of competency-based instruction that 
could be used to develop or serve as evaluation criteria for the project. 
The characteristics were identified with the assistance of the evaluation committee 
(project advisory committee). The identified characteristics were used in a Delphi 
. study to determine the required characteristics of a competency-based instruction 



program for this study. 
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4. Establish assessment criteria 

The characteristics determined by the Delphi will serve as the assessment criteria 
for this assessment. 

5. Develop and mail questionnaires to administrat ors and faculty 

(Components 4 through 7 of the conceptual model provides an outline for the plan 
for piloting the model). The assessment criteria (characteristics) will be used on 
the survey questionnaire to collect quantitive data. 

6. Compile and analyze response 

Responses to the survey questionnaires will be entered into computer storage 
when received. When all questionnaires are returned, a computer analysis will be 
made of the responses. 

7. Perform discrepancy analy sis 

Following the analysis of the response, a discrepancy analysis will be made. The 
algorithm 'or the discrepancy analysis is: 

WHAT SHOULD BE ± WHAT IS = DISCREPANCY 
(goals and objectives) (present condition) 

Since goals and objectives have not been specifically stated by the Coordinating 
Board, the characteristics of CBI established by the Delphi will serve as "What 
Should Be." The present condition of "What Is" will be the findings of the 
assessment. The "Discrepancy" will be the difference between a 100 percent 
implementation and the results of the assessment. 
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Identify strategies and resources and make recommendations to the Coordinating 
Board for the increase d implementation of competency-hased instruction on a 
statewide basis 
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Model for Local Evaluation 



1. Establish purpose of evaluation 

Most evaluations are conducted to determine program effectiveness in terms of 
the purpose of the program- The purpose of the program defines the expectations 
of the program in terms of use of funds, populations to be served, and services or 
products to be provided to the populations being served. 
If the purpose of the local evaluation is to determine the extent to which a 
competency-based instruction system has been implemented, the institution should 
use the state assessment model. The only exception in using the state assessment 
model at the institutional level is the survey procedure- At the institutional level, 
both surveys and interviews should be conducted with students, faculty, and 
administrators. 

2. Evaluation steering committee 

The steering committee should be composed of persons representatives of those 
groups affected by the competency-based instructional program: faculty, 
administrators, business and industry, and when appropriate, students. The normal 
role of the steering committee is to assist and advise in the evaluation process and 
to make recommendations to the appropriate administrative body based on the 
findings of the evaluation* For the annual local evaluation, the steering committee 
may also serve as the evaluators of the program(s). Specifically the steering 
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committee should address the following questions in developing the evaluation 
plan or scope of works to be performed: 
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1. Purpose: 

a. Is the evaluation being conducted to demonstrate 
accountability of the program(s) to the public for specified 
outcomes and results? 

b. Is the evaluation being conducted to determine the need for 
improvement of institutional procedures and activities? 

2. What program is to be evaluated? 

3. Types of evaluation: 

a. Will the evaluation be internal or external? 

b. Will there be active or passive involvement by college 
personnel? 

4. Evaluators: 

a. Who is to conduct the evaluation? 

b. What will be the roles and responsibilities of the evaluators? 

c. To what extent will there be an emphasis on formal 
evaluation designs using control groups and multiple criteria 
for measuring change? 

5. What types of information are to be collected for each 
evaluative criterion, program standard, or measure? 

6. Inquiring methods: 

a. What will be the emphasis on documents analysis, interviews, 
surveys? 

b. What will be the emphasis on observation? 

7. What will be the emphasis on quantitive and qualitative analysis 
of data collected? 

8. Who are the targeted audiences for the dissemination of the 
evaluation results? 
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Selection of evaluating criteria 

Perkins (PL 101-392) provides a very good design for accountability using annual 
evaluations. Perkins requires that applicants for Perkins funds "describe the 
program evaluation standards the applicant will use to measure its progress." 
Perkins requires that each recipient of financial assistance under part C of title II 
shall evaluate the effectiveness of the program(s) based on the core standards and 
measures established by the state agency administering the Perkins funds. 
The institution receiving Perkins funds must annually evaluate its technical/ 
vocational programs in ierms of: 

1. Progress 

2. Effectiveness 

3. ( Core standards and measures established by the Texas Higher 

Education Coordinating Board. The local institution has the 
prerogative of using additional standards and measures which 
they feel will assist the institution in better meeting the goals of 
the institution, the needs of students, and the needs of business 
and industry to be more competitive in the world economy. 

It is suggested that the core standards and performance measures established by 

the State be used in developing evaluation criteria for determining program 

effectiveness. 

It will be the responsibility of the local institutions to develop evaluation criteria 
or measures and standards for evaluating or measuring progress. 
A "Measure" as defined by Perkins is "a description of an outcome." A 
"Standard" as defined by Perkins is "the level or rate of an outcome." 
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The word "outcome" defined in WEBSTER'S DICTIONARY means 
"something that follows as a result or consequence." The EDUCATIONAL 
DICTIONARY, 1973, defines "outcome" as "change in behavior resulting from 
learning; not to be confused with objective, which is a desired result," 
If by definition an outcome is something that follows as a result or consequence, 
then a measure as defined by Perkins must be written in performance terms if it is 
to be used in evaluating progress or effectiveness, in developing criteria for 
summative evaluation of competency-based technical programs, it is recommended 
that both process evaluation and product evaluation be used in determining both 
progress and effectiveness of the program- 
Measures and standards for progress evaluation may be written for program and 
students in conducting the evaluation. Therefore, using Perkins definitions, the 
outcome or measure should be written in terms of what is expected of: (1) the 
students in relation to progress or competency attainment or (2) the program in 
relation to goals and objectives. 

There are several areas or components that should be considered when developing 
local standards and measures for evaluating progress of students in a competency- 
based instructional technical program. In most cases, local standards and 
measures will relate to students progress. However, there are other components 
that affect students progress and should be considered in the selection of the 
evaluation criteria. 

One of the institutional components that indirectly affect student progress or 



15 



112 



outcomes is student readiness to benefit from and achieve in the instructional 
program. Students who enter a program may not be routinely tested for 
communication, computational, and other skills that are prerequisites for the 
program or sequential courses of the program. These pre-enroilment assessments 
are necessary to ensure that each student is counseled in relation to academic 
achievements required by the program. The preliminary assessments are also 
needed to make instructors aware of special needs of students for remediation 
early in the course. The students performances on a preassessment should 
establish the criteria for placement in the selected sequence of courses of the 
program. 

Other components or areas that should be considered when developing evaluation 
standards and measures for progress evaluation include: 

• Student orientation 

• Instructional procedures 

• Instructor role 

• Competency exams 

• Progress records 

• Staff development 

• Curriculum development 

• Program Advisory Committees 

• Community relations 
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Evaluation based on student outcomes or achievement alone will not provide the 
information needed for making program improvements. Process information 
about the components that influence students progress should be included in the 
evaluation criteria, 

4. On - site visits 

On-site visits should be made of the program(s) to obtain data by observation and 
interview. On-site visits made by local evaluation may be formal in nature on a 
scheduled basis or on an unscheduled "drop-in" basis. The unscheduled on-site 
visits must comply with all of the institutional policies and procedures for 
visitation, 

5, Steering committee meetings 

After the steering committee has been created by appointment, someone other 
than an administrator should be selected to chair the committee. It will then be 
the responsibility of the chair to ensure that: 

• The committee has a published schedule of meetings and agenda 
items 

• The committee has appropriate representative membership 

• The plan or purpose of work for the evaluation is agreed upon 
by the committee; and 

• Minutes of the meetings are available for review 
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6. Conduct interviews with students, faculty, and administrators 

All of the data that needs to be collected during the evaluation cannot be 
effectively obtained by questionnaires. Information that is pertinent to the 
evaluation jannot always be obtained by the use of a "forced choice" on a survey 
form. Structured interviews should be developed and conducted to collect such 
desirable and useful data. The structured interviews may be conducted in person 
and/or by telephone. 

7. Survey students, faculty , and employers 

A major task in surveying for information is the selection or development of 
survey instruments appropriate for the data to be collected. Two factors related to 
this task are the: (1) selected standards and measures associated with people, 
program(s), and organizational structure, which are the sources for data collection; 
and (2) format of the instruments appropriate for specific information required for 
evaluation. Surveying involves the procedures of collecting and recording data. 
As a minimum, collection, procedures include: 

1. Observation — Educational researchers use direct observation to 
study or obtain information about the behaviors and interactions 
of students and faculty within the education environment. 

2. Questioning — Questioning is a means of collecting information 
by asking individuals about the measures being used in the 
evaluation. Questioning may also be used to determine 
knowledge about instructional content, processes, products, and 
perceptions. 



O 18 



3. Documentary Analysis — This data collection procedure focuses 
on activities that have occurred prior to the evaluation being 
conducted. Documents that may be used for sources of data in 
educational evaluation include: local, state, and federal reports; 
student records; class records; and program records. 

After the data collection procedures have been determined for the survey 

instrument format, then the type of recording procedure must be selected. Data 

to be collected for each measure or survey item may be recorded descriptively or 

numerically. Data recorded by description are qualitative and no attempt is made 

to assign a value to the properties described. 

Numerically recorded or defined data requires the assignment of numerical values 
to each measure or statement on the survey form. Numerical values are 
determined through measurement using some type of scale with specific definitions 
of the scale. 

The survey instruments may be designed using a single collection procedure and a 
single recording procedure. The instruments may be designed to use any 
combination of the three collection procedures and the two recordings procedures 
described above. 

The survey procedures and formats previously described may be used to collect 
and analyze: instructional materials and procedures; students progress and 
attainment of competencies, populations served, and other program records and 
documents. 
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8. Analyze findings 

The term "analysis" may be used in at least two different ways when referring to 
findings of the evaluation study, "Computer analysis" is the performance of 
mathematical and statistical procedures for the treatment of numerical data. 
The "visual analysis" of findings is performed by a review of all of the compiled 
data that are recorded for purposes of interpretation. The data collected and 
reported by the established measures (evaluative criteria) are compared with the 
established standards for the program or student progress and competency 
achievement. 

Basically the analysis of the findings for program evaluation follows the procedure 
of discrepancy analysis. The discrepancy analysis shows the difference in <4 what is" 
and the established "what should be." 

If mathematical/statistical procedures are to be performed on the data collected 
by a survey instrument, the procedures should be selected prior to finalization of 
the format of the survey form. 

9. Reporting evaluation results 

The design of the report should be completed prior to any data collection. The 
design of the report may influence the type of data to be collected and how data 
are collected. The report should: 

• Address the purpose of the evaluation; 

• Have a format that is directed to the targeted audience(s); 

• Present the results or findings of the evaluation in an attractive 
and informative display, and with a narrative that is clear and 



ERIC 1 ! 7 



easily understood by the intended audience; and 
♦ Include a section on conclusions and recommendations. 
It is recommended that an executive summary of the report be developed for 
dissemination to certain audiences. The executive summary should be of no more 
than ten pages in length and list the key findings and recommendations stated in 
the report. 

Oral reports, especially when presented with professionally prepared 
transparencies, can highlight the key findings, conclusions, and recommendations. 
The oral report provides an opportunity for immediate feedback. It also provides 
the opportunity for clarifications in response to questions from the audience and 
positive public relations that result from personal interchanges of information. 
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